Reward positivity affects temporal interval production in a continuous timing task.

The neural circuits of reward processing and interval timing (including the perception and production of temporal intervals) are functionally intertwined, suggesting that it might be possible for momentary reward processing to influence subsequent timing behavior. Previous animal and human studies have mainly focused on the effect of reward on interval perception, whereas its impact on interval production is less clear. In this study, we examined whether feedback, as an example of performance-contingent reward, biases interval production. We recorded EEG from 20 participants while they engaged in a continuous drumming task with different realistic tempos (1728 trials per participant). Participants received color-coded feedback after each beat about whether they were correct (on time) or incorrect (early or late). Regression-based EEG analysis was used to unmix the rapid occurrence of a feedback response called the reward positivity (RewP), which is traditionally observed in more slow-paced tasks. Using linear mixed modeling, we found that RewP amplitude predicted timing behavior for the upcoming beat. This performance-biasing effect of the RewP was interpreted as reflecting the impact of fluctuations in reward-related anterior cingulate cortex activity on timing, and the necessity of continuous paradigms to make such observations was highlighted.

The representation of priors and decisions in the human parietal cortex.

Animals actively sample their environment through orienting actions such as saccadic eye movements. Saccadic targets are selected based both on sensory evidence immediately preceding the saccade, and a "salience map" or prior built-up over multiple saccades. In the primate cortex, the selection of each individual saccade depends on competition between target-selective cells that ramp up their firing rate to saccade release. However, it is less clear how a cross-saccade prior might be implemented, either in neural firing or through an activity-silent mechanism such as modification of synaptic weights on sensory inputs. Here, we present evidence from magnetoencephalography for 2 distinct processes underlying the selection of the current saccade, and the representation of the prior, in human parietal cortex. While the classic ramping decision process for each saccade was reflected in neural firing rates (measured in the event-related field), a prior built-up over multiple saccades was implemented via modulation of the gain on sensory inputs from the preferred target, as evidenced by rapid frequency tagging. A cascade of computations over time (initial representation of the prior, followed by evidence accumulation and then an integration of prior and evidence) provides a mechanism by which a salience map may be built up across saccades in parietal cortex. It also provides insight into the apparent contradiction that inactivation of parietal cortex has been shown not to affect performance on single-trials, despite the presence of clear evidence accumulation signals in this region.

The neural correlates of continuous feedback processing.

Feedback processing is commonly studied by analyzing the brain's response to discrete rather than continuous events. Such studies have led to the hypothesis that rapid phasic midbrain dopaminergic activity tracks reward prediction errors (RPEs), the effects of which are measurable at the scalp via electroencephalography (EEG). Although studies using continuous feedback are sparse, recent animal work suggests that moment-to-moment changes in reward are tracked by slowly ramping midbrain dopaminergic activity. Some have argued that these ramping signals index state values rather than RPEs. Our goal here was to develop an EEG measure of continuous feedback processing in humans, then test whether its behavior could be accounted for by the RPE hypothesis. Participants completed a stimulus-response learning task in which a continuous reward cue gradually increased or decreased over time. A regression-based unmixing approach revealed EEG activity with a topography and time course consistent with the stimulus-preceding negativity (SPN), a scalp potential previously linked to reward anticipation and tonic dopamine release. Importantly, this reward-related activity depended on outcome expectancy: as predicted by the RPE hypothesis, activity for expected reward cues was reduced compared to unexpected reward cues. These results demonstrate the possibility of using human scalp-recorded potentials to track continuous feedback processing, and test candidate hypotheses of this activity.

A cognitive map for value-guided choice in ventromedial prefrontal cortex.

The prefrontal cortex is crucial for economic decision-making and representing the value of options. However, how such representations facilitate flexible decisions remains unknown. We reframe economic decision-making in prefrontal cortex in line with representations of structure within the medial temporal lobe because such cognitive map representations are known to facilitate flexible behaviour. Specifically, we framed choice between different options as a navigation process in value space. Here we show that choices in a 2D value space defined by reward magnitude and probability were represented with a grid-like code, analogous to that found in spatial navigation. The grid-like code was present in ventromedial prefrontal cortex (vmPFC) local field potential theta frequency and the result replicated in an independent dataset. Neurons in vmPFC similarly contained a grid-like code, in addition to encoding the linear value of the chosen option. Importantly, both signals were modulated by theta frequency - occurring at theta troughs but on separate theta cycles. Furthermore, we found sharp-wave ripples - a key neural signature of planning and flexible behaviour - in vmPFC, which were modulated by accuracy and reward. These results demonstrate that multiple cognitive map-like computations are deployed in vmPFC during economic decision-making, suggesting a new framework for the implementation of choice in prefrontal cortex.

Temporal scaling of human scalp-recorded potentials.

Much of human behavior is governed by common processes that unfold over varying timescales. Standard event-related potential analysis assumes fixed-duration responses relative to experimental events. However, recent single-unit recordings in animals have revealed neural activity scales to span different durations during behaviors demanding flexible timing. Here, we employed a general linear modeling approach using a combination of fixed-duration and variable-duration regressors to unmix fixed-time and scaled-time components in human magneto-/electroencephalography (M/EEG) data. We use this to reveal consistent temporal scaling of human scalp-recorded potentials across four independent electroencephalogram (EEG) datasets, including interval perception, production, prediction, and value-based decision making. Between-trial variation in the temporally scaled response predicts between-trial variation in subject reaction times, demonstrating the relevance of this temporally scaled signal for temporal variation in behavior. Our results provide a general approach for studying flexibly timed behavior in the human brain.

Novelty, uncertainty, and the looming horizon.

Novelty and uncertainty are powerful drivers of exploration that are often conflated. In this issue of Neuron, Cockburn and colleagues dissociate the two and report a key interaction: close to task termination, novel options appear much more attractive relative to uncertain options.

Task-level value affects trial-level reward processing.

Despite disagreement about how anterior cingulate cortex (ACC) supports decision making, a recent hypothesis suggests that activity in this region is best understood in the context of a task or series of tasks. One important task-level variable is average reward because it is both a known driver of effortful behaviour and an important determiner of the tasks in which we choose to engage. Here we asked how average task value affects reward-related ACC activity. To answer this question, we measured a reward-related signal said to be generated in ACC called the reward positivity (RewP) while participants gambled in three tasks of differing average value. The RewP was reduced in the high-value task, an effect that was not explainable by either reward magnitude or outcome expectancy. This result suggests that ACC does not evaluate outcomes and cues in isolation, but in the context of the value of the current task.

Behavioural and neural indices of perceptual decision-making in autistic children during visual motion tasks.

Many studies report atypical responses to sensory information in autistic individuals, yet it is not clear which stages of processing are affected, with little consideration given to decision-making processes. We combined diffusion modelling with high-density EEG to identify which processing stages differ between 50 autistic and 50 typically developing children aged 6-14 years during two visual motion tasks. Our pre-registered hypotheses were that autistic children would show task-dependent differences in sensory evidence accumulation, alongside a more cautious decision-making style and longer non-decision time across tasks. We tested these hypotheses using hierarchical Bayesian diffusion models with a rigorous blind modelling approach, finding no conclusive evidence for our hypotheses. Using a data-driven method, we identified a response-locked centro-parietal component previously linked to the decision-making process. The build-up in this component did not consistently relate to evidence accumulation in autistic children. This suggests that the relationship between the EEG measure and diffusion-modelling is not straightforward in autistic children. Compared to a related study of children with dyslexia, motion processing differences appear less pronounced in autistic children. Exploratory analyses also suggest weak evidence that ADHD symptoms moderate perceptual decision-making in autistic children.

Visual Motion and Decision-Making in Dyslexia: Reduced Accumulation of Sensory Evidence and Related Neural Dynamics.

Children with and without dyslexia differ in their behavioral responses to visual information, particularly when required to pool dynamic signals over space and time. Importantly, multiple processes contribute to behavioral responses. Here we investigated which processing stages are affected in children with dyslexia when performing visual motion processing tasks, by combining two methods that are sensitive to the dynamic processes leading to responses. We used a diffusion model which decomposes response time and accuracy into distinct cognitive constructs, and high-density EEG. Fifty children with dyslexia (24 male) and 50 typically developing children (28 male) 6-14 years of age judged the direction of motion as quickly and accurately as possible in two global motion tasks (motion coherence and direction integration), which varied in their requirements for noise exclusion. Following our preregistered analyses, we fitted hierarchical Bayesian diffusion models to the data, blinded to group membership. Unblinding revealed reduced evidence accumulation in children with dyslexia compared with typical children for both tasks. Additionally, we identified a response-locked EEG component which was maximal over centro-parietal electrodes which indicated a neural correlate of reduced drift rate in dyslexia in the motion coherence task, thereby linking brain and behavior. We suggest that children with dyslexia tend to be slower to extract sensory evidence from global motion displays, regardless of whether noise exclusion is required, thus furthering our understanding of atypical perceptual decision-making processes in dyslexia.SIGNIFICANCE STATEMENT Reduced sensitivity to visual information has been reported in dyslexia, with a lively debate about whether these differences causally contribute to reading difficulties. In this large preregistered study with a blind modeling approach, we combine state-of-the art methods in both computational modeling and EEG analysis to pinpoint the stages of processing that are atypical in children with dyslexia in two visual motion tasks that vary in their requirement for noise exclusion. We find reduced evidence accumulation in children with dyslexia across both tasks, and identify a neural marker, allowing us to link brain and behavior. We show that children with dyslexia exhibit general difficulties with extracting sensory evidence from global motion displays, not just in tasks that require noise exclusion.

Frontal circuit specialisations for decision making.

There is widespread consensus that distributed circuits across prefrontal and anterior cingulate cortex (PFC/ACC) are critical for reward-based decision making. The circuit specialisations of these areas in primates were likely shaped by their foraging niche, in which decision making is typically sequential, attention-guided and temporally extended. Here, I argue that in humans and other primates, PFC/ACC circuits are functionally specialised in two ways. First, microcircuits found across PFC/ACC are highly recurrent in nature and have synaptic properties that support persistent activity across temporally extended cognitive tasks. These properties provide the basis of a computational account of time-varying neural activity within PFC/ACC as a decision is being made. Second, the macrocircuit connections (to other brain areas) differ between distinct PFC/ACC cytoarchitectonic subregions. This variation in macrocircuit connections explains why PFC/ACC subregions make unique contributions to reward-based decision tasks and how these contributions are shaped by attention. They predict dissociable neural representations to emerge in orbitofrontal, anterior cingulate and dorsolateral prefrontal cortex during sequential attention-guided choice, as recently confirmed in neurophysiological recordings.

A Diversity of Intrinsic Timescales Underlie Neural Computations.

Neural processing occurs across a range of temporal scales. To facilitate this, the brain uses fast-changing representations reflecting momentary sensory input alongside more temporally extended representations, which integrate across both short and long temporal windows. The temporal flexibility of these representations allows animals to behave adaptively. Short temporal windows facilitate adaptive responding in dynamic environments, while longer temporal windows promote the gradual integration of information across time. In the cognitive and motor domains, the brain sets overarching goals to be achieved within a long temporal window, which must be broken down into sequences of actions and precise movement control processed across much shorter temporal windows. Previous human neuroimaging studies and large-scale artificial network models have ascribed different processing timescales to different cortical regions, linking this to each region's position in an anatomical hierarchy determined by patterns of inter-regional connectivity. However, even within cortical regions, there is variability in responses when studied with single-neuron electrophysiology. Here, we review a series of recent electrophysiology experiments that demonstrate the heterogeneity of temporal receptive fields at the level of single neurons within a cortical region. This heterogeneity appears functionally relevant for the computations that neurons perform during decision-making and working memory. We consider anatomical and biophysical mechanisms that may give rise to a heterogeneity of timescales, including recurrent connectivity, cortical layer distribution, and neurotransmitter receptor expression. Finally, we reflect on the computational relevance of each brain region possessing a heterogeneity of neuronal timescales. We argue that this architecture is of particular importance for sensory, motor, and cognitive computations.

Visual fixation patterns during economic choice reflect covert valuation processes that emerge with learning.

Visual fixations play a vital role in decision making. Recent studies have demonstrated that the longer subjects fixate an option, the more likely they are to choose it. However, the role of evaluating stimuli covertly (i.e., without fixating them), and how covert evaluations determine where to subsequently fixate, remains relatively unexplored. Here, we trained monkeys to perform a decision-making task where they made binary choices between reward-predictive stimuli which were well-learned ("overtrained"), recently learned ("novel"), or a combination of both ("mixed"). Subjects were free to saccade around the screen and make a choice (via joystick response) at any time. Subjects rarely fixated both options, yet choice behavior was better explained by assuming the values of both stimuli governed choices. The first fixation latency was fast (∼150 ms) but, surprisingly, its direction was value-driven. This suggests covert evaluation of stimulus values prior to first saccade. This was particularly evident for overtrained stimuli. For novel stimuli, first fixations became increasingly value-driven throughout a behavioral session. However, this improvement lagged behind learning of accurate economic choices, suggesting separate processes governed their learning. Finally, mixed trials revealed a strong bias toward fixating the novel stimulus first but no bias toward choosing it. Our results suggest that the primate brain contains fast covert evaluation mechanisms for guiding fixations toward highly valuable and novel information. By employing such covert mechanisms, fixation behavior becomes dissociable from the value comparison processes that drive final choice. This implies that primates use separable decision systems for value-guided fixations and value-guided choice.

Triple dissociation of attention and decision computations across prefrontal cortex.

Naturalistic decision-making typically involves sequential deployment of attention to choice alternatives to gather information before a decision is made. Attention filters how information enters decision circuits, thus implying that attentional control may shape how decision computations unfold. We recorded neuronal activity from three subregions of the prefrontal cortex (PFC) while monkeys performed an attention-guided decision-making task. From the first saccade to decision-relevant information, a triple dissociation of decision- and attention-related computations emerged in parallel across PFC subregions. During subsequent saccades, orbitofrontal cortex activity reflected the value comparison between currently and previously attended information. In contrast, the anterior cingulate cortex carried several signals reflecting belief updating in light of newly attended information, the integration of evidence to a decision bound and an emerging plan for what action to choose. Our findings show how anatomically dissociable PFC representations evolve during attention-guided information search, supporting computations critical for value-guided choice.

Neuroscience: Intracranial Recordings of Value.

The role of orbitofrontal cortex in value-based choice is well-established from animal research, but there are challenges in relating neurophysiological recordings from animals to equivalent data from humans: a new study bridges this gap.

Spontaneous cortical activity transiently organises into frequency specific phase-coupling networks.

Frequency-specific oscillations and phase-coupling of neuronal populations are essential mechanisms for the coordination of activity between brain areas during cognitive tasks. Therefore, the ongoing activity ascribed to the different functional brain networks should also be able to reorganise and coordinate via similar mechanisms. We develop a novel method for identifying large-scale phase-coupled network dynamics and show that resting networks in magnetoencephalography are well characterised by visits to short-lived transient brain states, with spatially distinct patterns of oscillatory power and coherence in specific frequency bands. Brain states are identified for sensory, motor networks and higher-order cognitive networks. The cognitive networks include a posterior alpha (8-12 Hz) and an anterior delta/theta range (1-7 Hz) network, both exhibiting high power and coherence in areas that correspond to posterior and anterior subdivisions of the default mode network. Our results show that large-scale cortical phase-coupling networks have characteristic signatures in very specific frequency bands, possibly reflecting functional specialisation at different intrinsic timescales.

Reconciling persistent and dynamic hypotheses of working memory coding in prefrontal cortex.

Competing accounts propose that working memory (WM) is subserved either by persistent activity in single neurons or by dynamic (time-varying) activity across a neural population. Here, we compare these hypotheses across four regions of prefrontal cortex (PFC) in an oculomotor-delayed-response task, where an intervening cue indicated the reward available for a correct saccade. WM representations were strongest in ventrolateral PFC neurons with higher intrinsic temporal stability (time-constant). At the population-level, although a stable mnemonic state was reached during the delay, this tuning geometry was reversed relative to cue-period selectivity, and was disrupted by the reward cue. Single-neuron analysis revealed many neurons switched to coding reward, rather than maintaining task-relevant spatial selectivity until saccade. These results imply WM is fulfilled by dynamic, population-level activity within high time-constant neurons. Rather than persistent activity supporting stable mnemonic representations that bridge subsequent salient stimuli, PFC neurons may stabilise a dynamic population-level process supporting WM.

Differential Valuation and Learning From Social and Nonsocial Cues in Borderline Personality Disorder.

BACKGROUND: Volatile interpersonal relationships are a core feature of borderline personality disorder (BPD) and lead to devastating disruption of patients' personal and professional lives. Quantitative models of social decision making and learning hold promise for defining the underlying mechanisms of this problem. In this study, we tested BPD and control subject weighting of social versus nonsocial information and their learning about choices under stable and volatile conditions. We compared behavior using quantitative models. METHODS: Subjects (n = 20 BPD, n = 23 control subjects) played an extended reward learning task with a partner (confederate) that requires learning about nonsocial and social cue reward probability (the social valuation task). Task experience was measured using language metrics: explicit emotions/beliefs, talk about the confederate, and implicit distress (using the previously established marker self-referentiality). Subjects' weighting of social and nonsocial cues was tested in mixed-effect regression models. Subjects' learning rates under stable and volatile conditions were modeled (Rescorla-Wagner approach) and group × condition interactions tested. RESULTS: Compared to control subjects, BPD subject debriefings included more mentions of the confederate and less distress language. BPD subjects also weighted social cues more heavily but had blunted learning responses to (nonsocial and social) volatility. CONCLUSIONS: This is the first report of patient behavior in the social valuation task. The results suggest that BPD subjects expect higher volatility than control subjects. These findings lay the groundwork for a neurocomputational dissection of social and nonsocial belief updating in BPD, which holds promise for the development of novel clinical interventions that more directly target pathophysiology.

Correction: Approach-Induced Biases in Human Information Sampling.

[This corrects the article DOI: 10.1371/journal.pbio.2000638.].

A distributed, hierarchical and recurrent framework for reward-based choice.

Many accounts of reward-based choice argue for distinct component processes that are serial and functionally localized. In this Opinion article, we argue for an alternative viewpoint, in which choices emerge from repeated computations that are distributed across many brain regions. We emphasize how several features of neuroanatomy may support the implementation of choice, including mutual inhibition in recurrent neural networks and the hierarchical organization of timescales for information processing across the cortex. This account also suggests that certain correlates of value are emergent rather than represented explicitly in the brain.