Mesostriatal dopamine is sensitive to changes in specific cue-reward contingencies.

Learning causal relationships relies on understanding how often one event precedes another. To investigate how dopamine neuron activity and neurotransmitter release change when a retrospective relationship is degraded for a specific pair of events, we used outcome-selective Pavlovian contingency degradation in rats. Conditioned responding was attenuated for the cue-reward contingency that was degraded, as was dopamine neuron activity in the midbrain and dopamine release in the ventral striatum in response to the cue and subsequent reward. Contingency degradation also abolished the trial-by-trial history dependence of the dopamine responses at the time of trial outcome. This profile of changes in cue- and reward-evoked responding is not easily explained by a standard reinforcement learning model. An alternative model based on learning causal relationships was better able to capture dopamine responses during contingency degradation, as well as conditioned behavior following optogenetic manipulations of dopamine during noncontingent rewards. Our results suggest that mesostriatal dopamine encodes the contingencies between meaningful events during learning.

Common neural dysfunction of economic decision-making across psychiatric conditions.

Adaptive decision-making, which is often impaired in various psychiatric conditions, is essential for well-being. Recent evidence has indicated that decision-making capacity in multiple tasks could be accounted for by latent dimensions, enlightening the question of whether there is a common disruption of brain networks in economic decision-making across psychiatric conditions. Here, we addressed the issue by combining activation/lesion network mapping analyses with a transdiagnostic brain imaging meta-analysis. Our findings indicate that there were transdiagnostic alterations in the thalamus and ventral striatum during the decision or outcome stage of decision-making. The identified regions represent key nodes in a large-scale network, which is composed of multiple heterogeneous brain regions and plays a causal role in motivational functioning. The findings suggest that disturbances in the network associated with emotion- and reward-related processing play a key role in dysfunctions of decision-making observed in various psychiatric conditions. This study provides the first meta-analytic evidence of common neural alterations linked to deficits in economic decision-making.

Exposure to community violence as a mechanism linking neighborhood socioeconomic disadvantage and neural responses to reward.

A growing literature links socioeconomic disadvantage and adversity to brain function, including disruptions in reward processing. Less research has examined exposure to community violence (ECV) as a specific adversity related to differences in reward-related brain activation, despite the prevalence of community violence exposure for those living in disadvantaged contexts. The current study tested whether ECV was associated with reward-related ventral striatum (VS) activation after accounting for familial factors associated with differences in reward-related activation (e.g. parenting and family income). Moreover, we tested whether ECV is a mechanism linking socioeconomic disadvantage to reward-related activation in the VS. We utilized data from 444 adolescent twins sampled from birth records and residing in neighborhoods with above-average levels of poverty. ECV was associated with greater reward-related VS activation, and the association remained after accounting for family-level markers of disadvantage. We identified an indirect pathway in which socioeconomic disadvantage predicted greater reward-related activation via greater ECV, over and above family-level adversity. These findings highlight the unique impact of community violence exposure on reward processing and provide a mechanism through which socioeconomic disadvantage may shape brain function.

Longitudinal neural and behavioral trajectories of charity contributions across adolescence.

This study examined the development of prosocial charity donations and neural activity in the ventral striatum when gaining rewards for self and for charity. Participants 10-22 years (95% European heritage) participated in three annual behavioral-fMRI waves (T1: n = 160, T2: n = 167, T3: n = 175). Behaviorally, donations to charity as measured with an economic Dictator Game increased with age. Perspective taking also increased with age. In contrast, self-gain and charity-gain enjoyment decreased with age. Ventral striatum activity was higher for rewards for self than for charity, but this difference decreased during adolescence. Latent growth curve models revealed that higher donations were associated with a smaller difference between ventral striatum activation for self and charity. These findings show longitudinal brain-donations associations in adolescence.

The effects of AUT00206, a novel Kv3.1/3.2 potassium channel modulator, on task-based reward system activation: a test of mechanism in schizophrenia.

The pathophysiology of schizophrenia involves abnormal reward processing, thought to be due to disrupted striatal and dopaminergic function. Consistent with this hypothesis, functional magnetic resonance imaging (fMRI) studies using the monetary incentive delay (MID) task report hypoactivation in the striatum during reward anticipation in schizophrenia. Dopamine neuron activity is modulated by striatal GABAergic interneurons. GABAergic interneuron firing rates, in turn, are related to conductances in voltage-gated potassium 3.1 (Kv3.1) and 3.2 (Kv3.2) channels, suggesting that targeting Kv3.1/3.2 could augment striatal function during reward processing. Here, we studied the effect of a novel potassium Kv3.1/3.2 channel modulator, AUT00206, on striatal activation in patients with schizophrenia, using the MID task. Each participant completed the MID during fMRI scanning on two occasions: once at baseline, and again following either 4 weeks of AUT00206 or placebo treatment. We found a significant inverse relationship at baseline between symptom severity and reward anticipation-related neural activation in the right associative striatum (r = -0.461, p = 0.035). Following treatment with AUT00206, there was a significant increase in reward anticipation-related activation in the left associative striatum (t(13) = 4.23, peak-level p(FWE) < 0.05)), but no significant effect in the ventral striatum. This provides preliminary evidence that the Kv3.1/3.2 potassium channel modulator, AUT00206, may address reward-related striatal abnormalities in schizophrenia.

The neurocomputational bases of explore-exploit decision-making.

Flexible decision-making requires animals to forego immediate rewards (exploitation) and try novel choice options (exploration) to discover if they are preferable to familiar alternatives. Using the same task and a partially observable Markov decision process (POMDP) model to quantify the value of choices, we first determined that the computational basis for managing explore-exploit tradeoffs is conserved across monkeys and humans. We then used fMRI to identify where in the human brain the immediate value of exploitative choices and relative uncertainty about the value of exploratory choices were encoded. Consistent with prior neurophysiological evidence in monkeys, we observed divergent encoding of reward value and uncertainty in prefrontal and parietal regions, including frontopolar cortex, and parallel encoding of these computations in motivational regions including the amygdala, ventral striatum, and orbitofrontal cortex. These results clarify the interplay between prefrontal and motivational circuits that supports adaptive explore-exploit decisions in humans and nonhuman primates.

Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior.

Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.

Long-term ecological assessment of intracranial electrophysiology synchronized to behavioral markers in obsessive-compulsive disorder.

Detection of neural signatures related to pathological behavioral states could enable adaptive deep brain stimulation (DBS), a potential strategy for improving efficacy of DBS for neurological and psychiatric disorders. This approach requires identifying neural biomarkers of relevant behavioral states, a task best performed in ecologically valid environments. Here, in human participants with obsessive-compulsive disorder (OCD) implanted with recording-capable DBS devices, we synchronized chronic ventral striatum local field potentials with relevant, disease-specific behaviors. We captured over 1,000 h of local field potentials in the clinic and at home during unstructured activity, as well as during DBS and exposure therapy. The wide range of symptom severity over which the data were captured allowed us to identify candidate neural biomarkers of OCD symptom intensity. This work demonstrates the feasibility and utility of capturing chronic intracranial electrophysiology during daily symptom fluctuations to enable neural biomarker identification, a prerequisite for future development of adaptive DBS for OCD and other psychiatric disorders.

Direct and indirect pathway neurons in ventrolateral striatum differentially regulate licking movement and nigral responses.

Drinking behavior in rodents is characterized by stereotyped, rhythmic licking movement, which is regulated by the basal ganglia. It is unclear how direct and indirect pathways control the lick bout and individual spout contact. We find that inactivating D1 and D2 receptor-expressing medium spiny neurons (MSNs) in the ventrolateral striatum (VLS) oppositely alters the number of licks in a bout. D1- and D2-MSNs exhibit different patterns of lick-sequence-related activity and different phases of oscillation time-locked to the lick cycle. On the timescale of a lick cycle, transient inactivation of D1-MSNs during tongue protrusion reduces spout contact probability, whereas transiently inactivating D2-MSNs has no effect. On the timescale of a lick bout, inactivation of D1-MSNs (D2-MSNs) causes rate increase (decrease) in a subset of basal ganglia output neurons that decrease firing during licking. Our results reveal the distinct roles of D1- and D2-MSNs in regulating licking at both coarse and fine timescales.

Segregating domain-general from emotional context-specific inhibitory control systems - ventral striatum and orbitofrontal cortex serve as emotion-cognition integration hubs.

Inhibitory control hierarchically regulates cognitive and emotional systems in the service of adaptive goal-directed behavior across changing task demands and environments. While previous studies convergently determined the contribution of prefrontal-striatal systems to general inhibitory control, findings on the specific circuits that mediate emotional context-specific impact on inhibitory control remained inconclusive. Against this background we combined an evaluated emotional Go/No Go task with fMRI in a large cohort of subjects (N=250) to segregate brain systems and circuits that mediate domain-general from emotion-specific inhibitory control. Particularly during a positive emotional context, behavioral results showed a lower accuracy for No Go trials and a faster response time for Go trials. While the dorsal striatum and lateral frontal regions were involved in inhibitory control irrespective of emotional context, activity in the ventral striatum (VS) and medial orbitofrontal cortex (mOFC) varied as a function of emotional context. On the voxel-wise whole-brain network level, limbic and striatal systems generally exhibited highest changes in global brain connectivity during inhibitory control, while global brain connectivity of the left mOFC was less decreased during emotional contexts. Functional connectivity analyses moreover revealed that negative coupling between the VS with inferior frontal gyrus (IFG)/insula and mOFC varied as a function of emotional context. Together these findings indicate separable domain- general as well as emotional context-specific inhibitory brain systems which specifically encompass the VS and its connections with frontal regions.

Anticipating control over aversive stimuli is mediated by the medial prefrontal cortex: An fMRI study with healthy adults.

The anticipation of control over aversive events in life is relevant for our mental health. Insights on the underlying neural mechanisms remain limited. We developed a new functional magnetic resonance imaging (fMRI) task that uses auditory stimuli to explore the neural correlates of (1) the anticipation of control over aversion and (2) the processing of aversion. In a sample of 25 healthy adults, we observed increased neural activation in the medial prefrontal cortex (ventromedial prefrontal cortex and rostral anterior cingulate cortex), other brain areas relevant for reward anticipation (ventral striatum, brainstem [ventral tegmental area], midcingulate cortex), and the posterior cingulate cortex when they anticipated control over aversion compared with anticipating no control (1). The processing of aversive sounds compared to neutral sounds (2) was associated with increased neural activation in the bilateral posterior insula. Our findings provide evidence for the important role of medial prefrontal regions in control anticipation and highlight the relevance of conceiving the neural mechanisms involved within a reward-based framework.

Better the devil you know than the devil you don't: Neural processing of risk and ambiguity.

Risk and ambiguity are inherent in virtually all human decision-making. Risk refers to a situation in which we know the precise probability of potential outcomes of each option, whereas ambiguity refers to a situation in which outcome probabilities are not known. A large body of research has shown that individuals prefer known risks to ambiguity, a phenomenon known as ambiguity aversion. One heated debate concerns whether risky and ambiguous decisions rely on the same or distinct neural circuits. In the current meta-analyses, we integrated the results of neuroimaging research on decision-making under risk (n = 69) and ambiguity (n = 31). Our results showed that both processing of risk and ambiguity showed convergence in anterior insula, indicating a key role of anterior insula in encoding uncertainty. Risk additionally engaged dorsomedial prefrontal cortex (dmPFC) and ventral striatum, whereas ambiguity specifically recruited the dorsolateral prefrontal cortex (dlPFC), inferior parietal lobe (IPL) and right anterior insula. Our findings demonstrate overlapping and distinct neural substrates underlying different types of uncertainty, guiding future neuroimaging research on risk-taking and ambiguity aversion.

Nalmefene attenuates neural alcohol cue-reactivity in the ventral striatum and subjective alcohol craving in patients with alcohol use disorder.

RATIONALE: Alcohol use disorder is a common and devastating mental illness for which satisfactory treatments are still lacking. Nalmefene, as an opioid receptor modulator, could pharmacologically support the reduction of drinking by reducing the (anticipated) rewarding effects of alcohol and expanding the range of treatment options. It has been hypothesized that nalmefene acts via an indirect modulation of the mesolimbic reward system. So far, only a few imaging findings on the neuronal response to nalmefene are available. OBJECTIVES: We tested the effect of a single dose of 18 mg nalmefene on neuronal cue-reactivity in the ventral and dorsal striatum and subjective craving. METHODS: Eighteen non-treatment-seeking participants with alcohol use disorder (67% male, M = 50.3 ± 13.9 years) with a current high-risk drinking level (M = 76.9 ± 52 g of pure alcohol per day) were investigated using a cue-reactivity task during functional magnetic resonance imaging (fMRI) in a double-blind, placebo-controlled, cross-over study/design. In addition, self-reported craving was assessed before and after exposure to alcohol cues. RESULTS: An a priori defined region of interest (ROI) analysis of fMRI data from 15 participants revealed that nalmefene reduced alcohol cue-reactivity in the ventral, but not the dorsal striatum. Additionally, the subjective craving was significantly reduced after the cue-reactivity task under nalmefene compared to placebo. CONCLUSION: In the present study, reduced craving and cue-reactivity to alcohol stimuli in the ventral striatum by nalmefene indicates a potential anti-craving effect of this drug via attenuation of neural alcohol cue-reactivity.

A bias towards natural rewards away from gambling cues in gamblers undergoing active treatment.

BACKGROUND: Disorders of substance and behavioral addiction are believed to be associated with a myopic bias towards the incentive salience of addiction-related cues away from general rewards in the environment. In non-treatment seeking gambling disorder patients, neural activity to anticipation of monetary rewards is enhanced relative to erotic rewards. Here we focus on the balance between anticipation of reward types in active treatment gamblers relative to healthy volunteers. METHODS: Fifty-three (25 gambling disorder males, 28 age-matched male healthy volunteers) were scanned with fMRI performing a Monetary Incentive Delay task with monetary and erotic outcomes. RESULTS: During reward anticipation, gambling disorder was associated with greater left orbitofrontal cortex and ventral striatal activity to erotic relative to monetary reward anticipation compared to healthy volunteers. Lower impulsivity correlated with greater activity in the dorsal striatum and dorsal anterior cingulate cortex to erotic anticipation in gambling disorder subjects. In the outcome phase, gambling disorder subjects showed greater activity in the ventral striatum, ventromedial and dorsolateral prefrontal cortex and anterior cingulate cortex to both reward types relative to healthy volunteers. CONCLUSIONS: These findings contrast directly with previous findings in non-treatment seeking gambling disorder. Our observations highlight the role of treatment state in active treatment gambling disorder, emphasizing a potential influence of treatment status, gambling abstinence or cognitive behavioral therapy on increasing the salience of general rewards beyond that of gambling-related cues. These findings support a potential therapeutic role for targeting the salience of non-gambling related rewards and potential biomarkers for treatment efficacy.

Medical education and distrust modulate the response of insular-cingulate network and ventral striatum in pain diagnosis.

Healthcare providers often underestimate patients' pain, sometimes even when aware of their reports. This could be the effect of experience reducing sensitivity to others pain, or distrust toward patients' self-evaluations. Across multiple experiments (375 participants), we tested whether senior medical students differed from younger colleagues and lay controls in the way they assess people's pain and take into consideration their feedback. We found that medical training affected the sensitivity to pain faces, an effect shown by the lower ratings and highlighted by a decrease in neural response of the insula and cingulate cortex. Instead, distrust toward the expressions' authenticity affected the processing of feedbacks, by decreasing activity in the ventral striatum whenever patients' self-reports matched participants' evaluations, and by promoting strong reliance on the opinion of other doctors. Overall, our study underscores the multiple processes which might influence the evaluation of others' pain at the early stages of medical career.

Primate ventral striatum maintains neural representations of the value of previously rewarded objects for habitual seeking.

The ventral striatum (VS) is considered a key region that flexibly updates recent changes in reward values for habit learning. However, this update process may not serve to maintain learned habitual behaviors, which are insensitive to value changes. Here, using fMRI in humans and single-unit electrophysiology in macaque monkeys we report another role of the primate VS: that the value memory subserving habitual seeking is stably maintained in the VS. Days after object-value associative learning, human and monkey VS continue to show increased responses to previously rewarded objects, even when no immediate reward outcomes are expected. The similarity of neural response patterns to each rewarded object increases after learning among participants who display habitual seeking. Our data show that long-term memory of high-valued objects is retained as a single representation in the VS and may be utilized to evaluate visual stimuli automatically to guide habitual behavior.

A ventral striatal prediction error signal in human fear extinction learning.

Animal studies have shown that the prediction error (PE) signal that drives fear extinction learning is encoded by phasic activity of midbrain dopamine (DA) neurons. Thus, the extinction PE resembles the appetitive PE that drives reward learning. In humans, fear extinction learning is less well understood. Using computational neuroimaging, a previous study from our group reported hemodynamic activity in the left ventral putamen, a subregion of the ventral striatum (VS), to correlate with a PE function derived from a formal associative learning model. The activity was modulated by genetic variation in a DA-related gene. To conceptually replicate and extend this finding, we here asked whether an extinction PE (EPE) signal in the left ventral putamen can also be observed when genotype information is not taken into account. Using an optimized experimental design for model estimation, we again observed EPE-related activity in the same striatal region, indicating that activation of this region is a feature of human extinction learning. We further observed significant EPE signals across wider parts of the VS as well as in frontal cortical areas. These results may suggest that the prediction errors during extinction learning are available to larger parts of the brain, as has also been observed in human neuroimaging studies of reward PE signaling. Conclusive evidence that the human EPE signal is of DAergic nature is still outstanding.

Friendship stability in adolescence is associated with ventral striatum responses to vicarious rewards.

An important task for adolescents is to form and maintain friendships. In this three-wave biannual study, we used a longitudinal neuroscience perspective to examine the dynamics of friendship stability. Relative to childhood and adulthood, adolescence is marked by elevated ventral striatum activity when gaining self-serving rewards. Using a sample of participants between the ages of eight and twenty-eight, we tested age-related changes in ventral striatum response to gaining for stable (n = 48) versus unstable best friends (n = 75) (and self). In participants with stable friendships, we observed a quadratic developmental trajectory of ventral striatum responses to winning versus losing rewards for friends, whereas participants with unstable best friends showed no age-related changes. Ventral striatum activity in response to winning versus losing for friends further varied with friendship closeness for participants with unstable friendships. We suggest that these findings may reflect changing social motivations related to formation and maintenance of friendships across adolescence.

Neural correlates of recursive thinking during interpersonal strategic interactions.

To navigate the complex social world, individuals need to represent others' mental states to think strategically and predict their next move. Strategic mentalizing can be classified into different levels of theory of mind according to its order of mental state attribution of other people's beliefs, desires, intentions, and so forth. For example, reasoning people's beliefs about simple world facts is the first-order attribution while going further to reason people's beliefs about the minds of others is the second-order attribution. The neural substrates that support such high-order recursive reasoning in strategic interpersonal interactions are still unclear. Here, using a sequential-move interactional game together with functional magnetic resonance imaging (fMRI), we showed that recursive reasoning engaged the frontal-subcortical regions. At the stimulus stage, the ventral striatum was more activated in high-order reasoning as compared with low-order reasoning. At the decision stage, high-order reasoning activated the medial prefrontal cortex (mPFC) and other mentalizing regions. Moreover, functional connectivity between the dorsomedial prefrontal cortex (dmPFC) and the insula/hippocampus was positively correlated with individual differences in high-order social reasoning. This work delineates the neural correlates of high-order recursive thinking in strategic games and highlights the key role of the interplay between mPFC and subcortical regions in advanced social decision-making.

Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory.

Recent behavioral evidence implicates reward prediction errors (RPEs) as a key factor in the acquisition of episodic memory. Yet, important neural predictions related to the role of RPEs in episodic memory acquisition remain to be tested. Humans (both sexes) performed a novel variable-choice task where we experimentally manipulated RPEs and found support for key neural predictions with fMRI. Our results show that in line with previous behavioral observations, episodic memory accuracy increases with the magnitude of signed (i.e., better/worse-than-expected) RPEs (SRPEs). Neurally, we observe that SRPEs are encoded in the ventral striatum (VS). Crucially, we demonstrate through mediation analysis that activation in the VS mediates the experimental manipulation of SRPEs on episodic memory accuracy. In particular, SRPE-based responses in the VS (during learning) predict the strength of subsequent episodic memory (during recollection). Furthermore, functional connectivity between task-relevant processing areas (i.e., face-selective areas) and hippocampus and ventral striatum increased as a function of RPE value (during learning), suggesting a central role of these areas in episodic memory formation. Our results consolidate reinforcement learning theory and striatal RPEs as key factors subtending the formation of episodic memory.SIGNIFICANCE STATEMENT Recent behavioral research has shown that reward prediction errors (RPEs), a key concept of reinforcement learning theory, are crucial to the formation of episodic memories. In this study, we reveal the neural underpinnings of this process. Using fMRI, we show that signed RPEs (SRPEs) are encoded in the ventral striatum (VS), and crucially, that SRPE VS activity is responsible for the subsequent recollection accuracy of one-shot learned episodic memory associations.