Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning.

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward rewards. Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) are crucial for this process, via engagement of a reciprocally connected network with their striatal targets. Critically, it remains unknown how dopamine neuron activity itself engages dopamine signals throughout the striatum, across learning. Here, we investigated how optogenetic Pavlovian cue conditioning of VTA or SNc dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. Although VTA stimulation-evoked robust dopamine release in NAc core, shell, and DMS, predictive cues preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS. Despite the development of vigorous movement late in training, conditioned dopamine signals did not emerge in the DLS, even during Pavlovian conditioning with SNc dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show a broad dissociation in the fundamental prediction and reward-related information generated by VTA and SNc dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale adaptations across the striatal network emerge during learning to coordinate behavior.

Valence ambiguity dynamically shapes striatal dopamine heterogeneity.

Adaptive decision making relies on dynamic updating of learned associations where environmental cues come to predict positive and negatively valenced stimuli, such as food or threat. Flexible cue-guided behaviors depend on a network of brain systems, including dopamine signaling in the striatum, which is critical for learning and maintenance of conditioned behaviors. Critically, it remains unclear how dopamine signaling encodes multi-valent, dynamic learning contexts, where positive and negative associations must be rapidly disambiguated. To understand this, we employed a Pavlovian discrimination paradigm, where cues predicting positive and negative outcomes were intermingled during conditioning sessions, and their meaning was serially reversed across training. We found that rats readily distinguished these cues, and updated their behavior rapidly upon valence reversal. Using fiber photometry, we recorded dopamine signaling in three major striatal subregions -,the dorsolateral striatum (DLS), the nucleus accumbens core, and the nucleus accumbens medial shell - and found heterogeneous responses to positive and negative conditioned cues and their predicted outcomes. Valence ambiguity introduced by cue reversal reshaped striatal dopamine on different timelines: nucleus accumbens core and shell signals updated more readily than those in the DLS. Together, these results suggest that striatal dopamine flexibly encodes multi-valent learning contexts, and these signals are dynamically modulated by changing contingencies to resolve ambiguity about the meaning of environmental cues.

Dopamine neurons drive spatiotemporally heterogeneous striatal dopamine signals during learning.

Environmental cues, through Pavlovian learning, become conditioned stimuli that invigorate and guide animals toward acquisition of rewards. Dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SNC) are crucial for this process. Dopamine neurons are embedded in a reciprocally connected network with their striatal targets, the functional organization of which remains poorly understood. Here, we investigated how learning during optogenetic Pavlovian cue conditioning of VTA or SNC dopamine neurons directs cue-evoked behavior and shapes subregion-specific striatal dopamine dynamics. We used a fluorescent dopamine biosensor to monitor dopamine in the nucleus accumbens (NAc) core and shell, dorsomedial striatum (DMS), and dorsolateral striatum (DLS). We demonstrate spatially heterogeneous, learning-dependent dopamine changes across striatal regions. While VTA stimulation evoked robust dopamine release in NAc core, shell, and DMS, cues predictive of this activation preferentially recruited dopamine release in NAc core, starting early in training, and DMS, late in training. Corresponding negative prediction error signals, reflecting a violation in the expectation of dopamine neuron activation, only emerged in the NAc core and DMS, and not the shell. Despite development of vigorous movement late in training, conditioned dopamine signals did not similarly emerge in the DLS, even during Pavlovian conditioning with SNC dopamine neuron activation, which elicited robust DLS dopamine release. Together, our studies show broad dissociation in the fundamental prediction and reward-related information generated by different dopamine neuron populations and signaled by dopamine across the striatum. Further, they offer new insight into how larger-scale plasticity across the striatal network emerges during Pavlovian learning to coordinate behavior.

Pavlovian cue-evoked alcohol seeking is disrupted by ventral pallidal inhibition.

Cues paired with alcohol can be potent drivers of craving, alcohol-seeking, consumption, and relapse. While the ventral pallidum is implicated in appetitive and consummatory responses across several reward classes and types of behaviors, its role in behavioral responses to Pavlovian alcohol cues has not previously been established. Here, we tested the impact of optogenetic inhibition of ventral pallidum on Pavlovian-conditioned alcohol-seeking in male Long Evans rats. Rats underwent Pavlovian conditioning with an auditory cue predicting alcohol delivery to a reward port and a control cue predicting no alcohol delivery, until they consistently entered the reward port more during the alcohol cue than the control cue. We then tested the within-session effects of optogenetic inhibition during 50% of cue presentations. We found that optogenetic inhibition of ventral pallidum during the alcohol cue reduced port entry likelihood and time spent in the port, and increased port entry latency. Overall, these results suggest that normal ventral pallidum activity is necessary for Pavlovian alcohol-seeking.

Basolateral amygdala population coding of a cued reward seeking state depends on orbitofrontal cortex.

Basolateral amygdala (BLA) neuronal responses to conditioned stimuli are closely linked to the expression of conditioned behavior. An area of increasing interest is how the dynamics of BLA neurons relate to evolving behavior. Here, we recorded the activity of individual BLA neurons across the acquisition and extinction of conditioned reward seeking and employed population-level analyses to assess ongoing neural dynamics. We found that, with training, sustained cue-evoked activity emerged that discriminated between the CS+ and CS- and correlated with conditioned responding. This sustained population activity continued until reward receipt and rapidly extinguished along with conditioned behavior during extinction. To assess the contribution of orbitofrontal cortex (OFC), a major reciprocal partner to BLA, to this component of BLA neural activity, we inactivated OFC while recording in BLA and found blunted sustained cue-evoked activity in BLA that accompanied reduced reward seeking. Optogenetic disruption of BLA activity and OFC terminals in BLA also reduced reward seeking. Our data suggest that sustained cue-driven activity in BLA, which in part depends on OFC input, underlies conditioned reward-seeking states.

Sensory Cues Potentiate VTA Dopamine Mediated Reinforcement.

Sensory cues are critical for shaping decisions and invigorating actions during reward seeking. Dopamine neurons in the ventral tegmental area (VTA) are central in this process, supporting associative learning in Pavlovian and instrumental settings. Studies of intracranial self-stimulation (ICSS) behavior, which show that animals will work hard to receive stimulation of dopamine neurons, support the notion that dopamine transmits a reward or value signal to support learning. Recent studies have begun to question this, however, emphasizing dopamine's value-free functions, leaving its contribution to behavioral reinforcement somewhat muddled. Here, we investigated the role of sensory stimuli in dopamine-mediated reinforcement, using an optogenetic ICSS paradigm in tyrosine hydroxylase (TH)-Cre rats. We find that while VTA dopamine neuron activation in the absence of explicit external cues is sufficient to maintain robust self-stimulation, the presence of cues dramatically potentiates ICSS behavior. Our results support a framework where dopamine can have some base value as a reinforcer, but the impact of this signal is modulated heavily by the sensory learning context.

Contexts facilitate dynamic value encoding in the mesolimbic dopamine system.

Adaptive behavior in a dynamic environment often requires rapid revaluation of stimuli that deviates from well-learned associations. The divergence between stable value-encoding and appropriate behavioral output remains a critical test to theories of dopamine's function in learning, motivation, and motor control. Yet how dopamine neurons are involved in the revaluation of cues when the world changes to alter our behavior remains unclear. Here we make use of pharmacology, in vivo electrophysiology, fiber photometry, and optogenetics to resolve the contributions of the mesolimbic dopamine system to the dynamic reorganization of reward-seeking. Male and female rats were trained to discriminate when a conditioned stimulus would be followed by sucrose reward by exploiting the prior, non-overlapping presentation of a separate discrete cue - an occasion setter. Only when the occasion setter's presentation preceded the conditioned stimulus did the conditioned stimulus predict sucrose delivery. As a result, in this task we were able to dissociate the average value of the conditioned stimulus from its immediate expected value on a trial-to-trial basis. Both the activity of ventral tegmental area dopamine neurons and dopamine signaling in the nucleus accumbens were essential for rats to successfully update behavioral responding in response to the occasion setter. Moreover, dopamine release in the nucleus accumbens following the conditioned stimulus only occurred when the occasion setter indicated it would predict reward. Downstream of dopamine release, we found that single neurons in the nucleus accumbens dynamically tracked the value of the conditioned stimulus. Together these results reveal a novel mechanism within the mesolimbic dopamine system for the rapid revaluation of motivation.

Sensory cues potentiate VTA dopamine mediated reinforcement.

Sensory cues are critical for shaping decisions and invigorating actions during reward seeking. Dopamine neurons in the ventral tegmental area (VTA) are critical in this process, supporting associative learning in Pavlovian and instrumental settings. Studies of intracranial self stimulation (ICSS) behavior, which show that animals will work hard to receive stimulation of dopamine neurons, support the notion that dopamine transmits a reward or value signal to support learning. Recent studies have begun to question this, however, emphasizing dopamine's value-free functions, leaving its contribution to behavioral reinforcement somewhat muddled. Here, we investigated the role of sensory stimuli in dopamine-mediated reinforcement, using an optogenetic ICSS paradigm in tyrosine hydroxylase (TH)-cre rats. We find that while VTA dopamine neuron activation in the absence of any external cueing stimulus is sufficient to maintain robust self stimulation, the presence of cues dramatically potentiates ICSS behavior. Our results support a framework where dopamine can have some base value as a reinforcer, but the impact of this signal is modulated heavily by the sensory learning context.

Hierarchical cue control of cocaine seeking in the face of cost.

RATIONALE: Addiction is characterized by intermittent drug seeking despite rising costs. This behavior is heavily influenced by environmental stimuli that signal drug availability and reinforce drug seeking. OBJECTIVE: To establish the relationship between three key aspects of human drug use in rats: the intermittent, binge nature of drug intake, the motivational conflict of drug seeking in the face of escalating negative costs, and the ability of different drug cues to interact to modulate relapse. METHODS: Male and female rats were trained to self-administer cocaine on an intermittent access schedule, where brief drug-availability states were signaled by a shift in the ambient lighting of the environment, and cocaine infusions were signaled by a separate proximal discrete cue. Rats then went through a conflict procedure, where foot shock intensity associated with cocaine seeking was escalated until intake was suppressed. We then completed relapse tests where the drug-delivery cue was noncontingently presented alone, or in the context of dynamic drug-availability state transitions. RESULTS: Intermittent access spurred psychomotor sensitization and binge-like cocaine intake. The intensity of binge-like drug taking during training was predictive of later drug seeking despite escalating costs during conflict. In relapse tests, the ability of a proximal discrete drug cue to trigger relapse was gated by the presence of a global cue signaling drug-availability state transitions. CONCLUSIONS: Our results suggest that the pattern of drug intake plays a role in many features of addiction, including modifying an individual's willingness to endure high costs associated with drug seeking. Furthermore, our studies indicate that drug-related sensory information can be hierarchically organized to exert a dynamic modulating influence on drug-seeking motivation.

Dopamine Circuit Mechanisms of Addiction-Like Behaviors.

Addiction is a complex disease that impacts millions of people around the world. Clinically, addiction is formalized as substance use disorder (SUD), with three primary symptom categories: exaggerated substance use, social or lifestyle impairment, and risky substance use. Considerable efforts have been made to model features of these criteria in non-human animal research subjects, for insight into the underlying neurobiological mechanisms. Here we review evidence from rodent models of SUD-inspired criteria, focusing on the role of the striatal dopamine system. We identify distinct mesostriatal and nigrostriatal dopamine circuit functions in behavioral outcomes that are relevant to addictions and SUDs. This work suggests that striatal dopamine is essential for not only positive symptom features of SUDs, such as elevated intake and craving, but also for impairments in decision making that underlie compulsive behavior, reduced sociality, and risk taking. Understanding the functional heterogeneity of the dopamine system and related networks can offer insight into this complex symptomatology and may lead to more targeted treatments.

Heterogeneity in striatal dopamine circuits: Form and function in dynamic reward seeking.

The striatal dopamine system has long been studied in the context of reward learning, motivation, and movement. Given the prominent role dopamine plays in a variety of adaptive behavioral states, as well as diseases like addiction, it is essential to understand the full complexity of dopamine neurons and the striatal systems they target. A growing number of studies are uncovering details of the heterogeneity in dopamine neuron subpopulations. Here, we review that work to synthesize current understanding of dopamine system heterogeneity across three levels, anatomical organization, functions in behavior, and modes of action, wherein we focus on signaling profiles and local mechanisms for modulation of dopamine release. Together, these studies reveal new and emerging dimensions of the striatal dopamine system, informing its contribution to dynamic motivational and decision-making processes.

Ring of Power: A Band of Peptidergic Midbrain Neurons that Binds Motivation.

A recent Cell paper identifies a novel population of neurons within the ventral tegmental area producing the endogenous opioid nociceptin that regulates dopamine neuron firing and acts uniquely to gate motivation in reward seeking. These results highlight neuropeptidergic signaling as a critical component of functional heterogeneity in the midbrain.

Dopamine neurons create Pavlovian conditioned stimuli with circuit-defined motivational properties.

Environmental cues, through Pavlovian learning, become conditioned stimuli that guide animals toward the acquisition of rewards (for example, food) that are necessary for survival. We tested the fundamental role of midbrain dopamine neurons in conferring predictive and motivational properties to cues, independent of external rewards. We found that brief phasic optogenetic excitation of dopamine neurons, when presented in temporal association with discrete sensory cues, was sufficient to instantiate those cues as conditioned stimuli that subsequently both evoked dopamine neuron activity on their own and elicited cue-locked conditioned behavior. Notably, we identified highly parcellated functions for dopamine neuron subpopulations projecting to different regions of striatum, revealing dissociable dopamine systems for the generation of incentive value and conditioned movement invigoration. Our results indicate that dopamine neurons orchestrate Pavlovian conditioning via functionally heterogeneous, circuit-specific motivational signals to create, gate, and shape cue-controlled behaviors.

Contemporary approaches to neural circuit manipulation and mapping: focus on reward and addiction.

Tying complex psychological processes to precisely defined neural circuits is a major goal of systems and behavioural neuroscience. This is critical for understanding adaptive behaviour, and also how neural systems are altered in states of psychopathology, such as addiction. Efforts to relate psychological processes relevant to addiction to activity within defined neural circuits have been complicated by neural heterogeneity. Recent advances in technology allow for manipulation and mapping of genetically and anatomically defined neurons, which when used in concert with sophisticated behavioural models, have the potential to provide great insight into neural circuit bases of behaviour. Here we discuss contemporary approaches for understanding reward and addiction, with a focus on midbrain dopamine and cortico-striato-pallidal circuits.

Sign-tracking to an appetitive cue predicts incubation of conditioned fear in rats.

Although post-traumatic stress disorder (PTSD) and addiction are very different disorders, both are characterized by hyperreactivity to trauma- or drug-related cues, respectively. We investigated whether an appetitive conditioning task, Pavlovian conditioned approach, which predicts vulnerability to reinstatement of cocaine-seeking, also predicts fear incubation, which may be a marker for vulnerability to PTSD. We classified rats based on whether they learned to approach and interact with a food predictive cue (sign-trackers), or, whether upon cue presentation they went to the location of impending food delivery (goal-trackers). Rats were then exposed to extensive Pavlovian tone-shock pairings, which causes the fear response to increase or "incubate" over time. We found that the fear incubation effect was only present in sign-trackers. The behavior of goal-trackers was more consistent with a normal fear response-it was most robust immediately after training and decayed slowly over time. Sign-trackers also had lower levels of brain-derived neurotrophic factor (BDNF) protein in the prefrontal cortex than goal-trackers. These results indicate that, while many factors likely contribute to the disproportionate co-occurrence of PTSD and substance abuse, one such factor may be a core psychological trait that biases some individuals to attribute excessive motivational significance to predictive cues, regardless of the emotional valence of those cues. High levels of BDNF in the prefrontal cortex may be protective against developing excessive emotional and motivational responses to salient cues.

A cocaine context renews drug seeking preferentially in a subset of individuals.

Addiction is characterized by a high propensity for relapse, in part because cues associated with drugs can acquire Pavlovian incentive motivational properties, and acting as incentive stimuli, such cues can instigate and invigorate drug-seeking behavior. There is, however, considerable individual variation in the propensity to attribute incentive salience to reward cues. Discrete and localizable reward cues act as much more effective incentive stimuli in some rats ('sign-trackers', STs), than others ('goal-trackers', GTs). We asked whether similar individual variation exists for contextual cues associated with cocaine. Cocaine context conditioned motivation was quantified in two ways: (1) the ability of a cocaine context to evoke conditioned hyperactivity and (2) the ability of a context in which cocaine was previously self-administered to renew cocaine-seeking behavior. Finally, we assessed the effects of intra-accumbens core flupenthixol, a nonselective dopamine receptor antagonist, on context renewal. In contrast to studies using discrete cues, a cocaine context spurred greater conditioned hyperactivity, and more robustly renewed extinguished cocaine seeking in GTs than STs. In addition, cocaine context renewal was blocked by antagonism of dopamine receptors in the accumbens core. Thus, contextual cues associated with cocaine preferentially acquire motivational control over behavior in different individuals than do discrete cues, and in these individuals the ability of a cocaine context to create conditioned motivation for cocaine requires dopamine in the core of the nucleus accumbens. We speculate that different individuals may be preferentially sensitive to different 'triggers' of relapse.

Positive reinforcement mediated by midbrain dopamine neurons requires D1 and D2 receptor activation in the nucleus accumbens.

The neural basis of positive reinforcement is often studied in the laboratory using intracranial self-stimulation (ICSS), a simple behavioral model in which subjects perform an action in order to obtain exogenous stimulation of a specific brain area. Recently we showed that activation of ventral tegmental area (VTA) dopamine neurons supports ICSS behavior, consistent with proposed roles of this neural population in reinforcement learning. However, VTA dopamine neurons make connections with diverse brain regions, and the specific efferent target(s) that mediate the ability of dopamine neuron activation to support ICSS have not been definitively demonstrated. Here, we examine in transgenic rats whether dopamine neuron-specific ICSS relies on the connection between the VTA and the nucleus accumbens (NAc), a brain region also implicated in positive reinforcement. We find that optogenetic activation of dopaminergic terminals innervating the NAc is sufficient to drive ICSS, and that ICSS driven by optical activation of dopamine neuron somata in the VTA is significantly attenuated by intra-NAc injections of D1 or D2 receptor antagonists. These data demonstrate that the NAc is a critical efferent target sustaining dopamine neuron-specific ICSS, identify receptor subtypes through which dopamine acts to promote this behavior, and ultimately help to refine our understanding of the neural circuitry mediating positive reinforcement.

On the motivational properties of reward cues: Individual differences.

Cues associated with rewards, such as food or drugs of abuse, can themselves acquire motivational properties. Acting as incentive stimuli, such cues can exert powerful control over motivated behavior, and in the case of cues associated with drugs, they can goad continued drug-seeking behavior and relapse. However, recent studies reviewed here suggest that there are large individual differences in the extent to which food and drug cues are attributed with incentive salience. Rats prone to approach reward cues (sign-trackers) attribute greater motivational value to discrete localizable cues and interoceptive cues than do rats less prone to approach reward cues (goal-trackers). In contrast, contextual cues appear to exert greater control over motivated behavior in goal-trackers than sign-trackers. It is possible to predict, therefore, before any experience with drugs, in which animals specific classes of drug cues will most likely reinstate drug-seeking behavior. The finding that different individuals may be sensitive to different triggers capable of motivating behavior and producing relapse suggests there may be different pathways to addiction, and has implications for thinking about individualized treatment. This article is part of a Special Issue entitled 'NIDA 40th Anniversary Issue'.