Effects of psychedelic, DOI, on nucleus accumbens dopamine signaling to predictable rewards and cues in rats.

Psychedelics produce lasting therapeutic responses in neuropsychiatric diseases suggesting they may disrupt entrenched associations and catalyze learning. Here, we examine psychedelic 5-HT2A/2C agonist, DOI, effects on dopamine signaling in the nucleus accumbens (NAc) core, a region extensively linked to reward learning, motivation, and drug-seeking. We measure phasic dopamine transients following acute DOI administration in rats during well learned Pavlovian tasks in which sequential cues predict rewards. We find that DOI (0.0-1.2 mg/kg, i.p.) increases dopamine signals, photometrically measured using GRABDA optical sensor, to rewards and proximal reward cues, but not to the distal cues that predict these events. We determine that the elevated dopamine produced by DOI to reward cues occurs independently of DOI-induced changes in reward value. The increased dopamine associated with predictable reward cues and rewards supports DOI-induced increases in prediction error signaling. These findings lay a foundation for developing psychedelic strategies aimed at engaging error-driven learning mechanisms to disrupt entrenched associations or produce new associations.

Sex specific effects of dorsomedial striatal cannabinoid receptor-1 signaling on Pavlovian outcome devaluation.

Cannabinoid-1 receptor (CB1R) signaling in the dorsal striatum regulates the shift from flexible to habitual behavior in instrumental outcome devaluation. Based on prior work establishing individual, sex, and experience-dependent differences in Pavlovian behaviors, we predicted a role for dorsomedial striatum CB1R signaling in driving sign-tracking and rigid responding in Pavlovian outcome devaluation. We trained male and female rats in Pavlovian Lever Autoshaping to determine sign-, or goal- or intermediate tracking groups. After extended Pavlovian training, we gave intra-DMS infusions of the CB1R inverse agonist, rimonabant, before satiety-induced outcome devaluation test sessions, in which we sated rats on training pellets (devalued) or home cage chow (valued) and examined responding to cues in brief nonreinforced Pavlovian Lever Autoshaping sessions. Overall, DMS CB1R signaling inhibition blocked Pavlovian outcome devaluation. After extended training, male rats were sensitive to devaluation while female rats were not. Inhibition of DMS CB1R signaling impaired Pavlovian outcome devaluation in male sign-tracking rats making their behavior more rigid but had no effects in female sign-tracking rats. Intra-DMS rimonabant infusions before reinforced sessions had no effects on Pavlovian sign- or goal-tracking in either sex. The sex-specific and CB1R-dependent effects were specific to outcome devaluation and were consistent between sign- and goal-tracking groups. Our results demonstrate that DMS endocannabinoid receptor signaling regulates behavioral flexibility in a sex-specific manner, suggesting differences in CB1R signaling in DMS between male and female rats.

Decreased Ventral Tegmental Area CB1R Signaling Reduces Sign Tracking and Shifts Cue-Outcome Dynamics in Rat Nucleus Accumbens.

Sign-tracking (ST) rats show enhanced cue sensitivity before drug experience that predicts greater discrete cue-induced drug seeking compared with goal-tracking or intermediate rats. Cue-evoked dopamine in the nucleus accumbens (NAc) is a neurobiological signature of sign-tracking behaviors. Here, we examine a critical regulator of the dopamine system, endocannabinoids, which bind the cannabinoid receptor-1 (CB1R) in the ventral tegmental area (VTA) to control cue-evoked striatal dopamine levels. We use cell type-specific optogenetics, intra-VTA pharmacology, and fiber photometry to test the hypothesis that VTA CB1R receptor signaling regulates NAc dopamine levels to control sign tracking. We trained male and female rats in a Pavlovian lever autoshaping (PLA) task to determine their tracking groups before testing the effect of VTA → NAc dopamine inhibition. We found that this circuit is critical for mediating the vigor of the ST response. Upstream of this circuit, intra-VTA infusions of rimonabant, a CB1R inverse agonist, during PLA decrease lever and increase food cup approach in sign-trackers. Using fiber photometry to measure fluorescent signals from a dopamine sensor, GRABDA (AAV9-hSyn-DA2m), we tested the effects of intra-VTA rimonabant on NAc dopamine dynamics during autoshaping in female rats. We found that intra-VTA rimonabant decreased sign-tracking behaviors, which was associated with increases in NAc shell, but not core, dopamine levels during reward delivery [unconditioned stimulus (US)]. Our results suggest that CB1R signaling in the VTA influences the balance between the conditioned stimulus-evoked and US-evoked dopamine responses in the NAc shell and biases behavioral responding to cues in sign-tracking rats.SIGNIFICANCE STATEMENT Substance use disorder (SUD) is a chronically relapsing psychological disorder that affects a subset of individuals who engage in drug use. Recent research suggests that there are individual behavioral and neurobiological differences before drug experience that predict SUD and relapse vulnerabilities. Here, we investigate how midbrain endocannabinoids regulate a brain pathway that is exclusively involved in driving cue-motivated behaviors of sign-tracking rats. This work contributes to our mechanistic understanding of individual vulnerabilities to cue-triggered natural reward seeking that have relevance for drug-motivated behaviors.

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related.

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

Inactivation of the Basolateral Amygdala to Insular Cortex Pathway Makes Sign-Tracking Sensitive to Outcome Devaluation.

Goal-tracking (GT) rats are sensitive to Pavlovian outcome devaluation while sign-tracking (ST) rats are devaluation insensitive. During outcome devaluation, GT rats flexibly modify responding to cues based on the current value of the associated outcome. However, ST rats rigidly respond to cues regardless of the current outcome value. Prior work demonstrated disconnection of the basolateral amygdala (BLA) and anterior insular cortex (aIC) decreased both GT and ST behaviors. Given the role of these regions in appetitive motivation and behavioral flexibility, we predicted that disrupting BLA to aIC pathway during outcome devaluation would reduce flexibility in GT rats and reduce rigid appetitive motivation in ST rats. We inhibited the BLA to aIC pathway by infusing inhibitory DREADDs (hM4Di-mcherry) or control (mCherry) virus into the BLA and implanted cannulae into the aIC to inhibit BLA terminals using intracranial injections of clozapine N-oxide (CNO). After training, we used a within-subject satiety-induced outcome devaluation procedure in which we sated rats on training pellets (devalued condition) or homecage chow (valued condition). All rats received bilateral CNO infusions into the aIC before brief nonreinforced test sessions. Contrary to our hypothesis, BLA-IC inhibition did not interfere with devaluation sensitivity in GT rats but did make ST behaviors sensitive to devaluation. Intermediate rats showed the opposite effect, showing rigid responding to cues with BLA-aIC pathway inactivation. Together, these results demonstrate BLA-IC projections mediate tracking-specific Pavlovian devaluation sensitivity and highlights the importance of considering individual differences in Pavlovian approach when evaluating circuitry contributions to behavioral flexibility.

Investigating discriminative stimulus modulation of opioid seeking after conflict-induced abstinence in sign- and goal-tracking rats.

RATIONALE: Discriminative stimuli (DS) are cues that predict reward availability. DS are resistant to extinction and motivate drug seeking even after long periods of abstinence. Previous studies have demonstrated that sign-tracking (ST) and goal-tracking (GT) differences in Pavlovian approach predict distinct cue-modulated vulnerabilities to cocaine reinstatement. GT rats show heightened reinstatement to contextual and DS, while ST rats show heightened reinstatement to discrete stimuli. Here we examine whether DS modulate reinstatement after electric barrier-induced abstinence and whether tracking-related relapse vulnerabilities generalize to opioid relapse. OBJECTIVES: We examine whether DS-modulated reinstatement to fentanyl seeking persists in the presence of reduced adverse consequences after electric barrier-induced abstinence. We also examine whether tracking differences predict the magnitude of DS-modulated reinstatement of fentanyl seeking after electric barrier-induced abstinence. METHODS: We used Pavlovian lever autoshaping (PLA) training to determine sign-, goal-, and intermediate tracking groups in male and female Sprague Dawley rats. We then trained rats in a DS model of intermittent fentanyl self-administration, and extinguished drug seeking by imposing an electric barrier of increasing intensity. We then measured the level of DS-modulated reinstatement in the presence of a reduced electric barrier intensity. RESULTS: We report that DS strongly modulate fentanyl seeking after electric barrier-induced abstinence. DS-modulation of fentanyl acquisition, electric barrier-induced abstinence, and reinstatement was similar for sign- and goal-tracking groups. CONCLUSIONS: Discriminative stimuli powerfully motivate opioid seeking, despite continued aversive consequences. Pavlovian approach differences do not predict the level of DS-modulated reinstatement to fentanyl seeking after conflict-induced abstinence.

Corrigendum: Role of BNST CRFR1 Receptors in Incubation of Fentanyl Seeking.

[This corrects the article DOI: 10.3389/fnbeh.2020.00153.].

Choose your path: Divergent basolateral amygdala efferents differentially mediate incentive motivation, flexibility and decision-making.

To survive in a complex environment, individuals form associations between environmental stimuli and rewards to organize and optimize reward seeking behaviors. The basolateral amygdala (BLA) uses these learned associations to inform decision-making processes. In this review, we describe functional projections between BLA and its cortical and striatal targets that promote learning and motivational processes central to decision-making. Specifically, we compare and contrast divergent projections from the BLA to the orbitofrontal (OFC) and to the nucleus accumbens (NAc) and examine the roles of these pathways in associative learning, value-guided decision-making, choice behaviors, as well as cue and context-driven drug seeking. Finally, we consider how these projections are involved in disorders of motivation, with a focus on Substance Use Disorder.

Effects of 5-HT2A receptor stimulation on economic demand for fentanyl after intermittent and continuous access self-administration in male rats.

The relative value of and motivation for abused drugs often increases with drug experience and differs based on drug availability. Here, we determined how different intake patterns of fentanyl, a µ-opioid agonist, alter economic demand for fentanyl and how 5-HT2A receptor stimulation affects economic demand for fentanyl. We used a within-session demand threshold procedure to characterize changes in economic demand for fentanyl before and after intermittent or continuous access schedules. We subsequently tested the acute effects of 5-HT2A receptor stimulation with psychedelic 2,5-dimethoxy-4-iodoamphetamine (DOI) on economic demand for fentanyl. Extended fentanyl experience with both intermittent and continuous schedules increased fentanyl consumption at low cost (Q0 ), increased total fentanyl consumption, and decreased demand elasticity (α), indicating both schedules elevated motivation to self-administer fentanyl. Overall, the two schedules produced similar alterations in economic demand for fentanyl, although low-cost consumption (Q0 ) increased more in the continuous access group. Systemic injections of DOI (0.0-0.4 mg/kg, i.p.) dose-dependently produced economic demand changes in the opposite direction produced by fentanyl experience. DOI decreased motivation (increased "α"), decreased Q0 , and decreased total fentanyl consumption. The selective 5-HT2A antagonist, M100907 (0.3 mg/kg, i.p.), blocked the effects of DOI, indicating that DOI is acting through 5-HT2A receptors to alter economic demand for fentanyl. In an economic food demand experiment, DOI (0.4 mg/kg) also increased demand elasticity and reduced food consumption. These results demonstrate that both intermittent and continuous fentanyl experience raise the economic demand for fentanyl, and acute 5-HT2A receptor activation reduces economic demand for fentanyl and food.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats.

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Role of BNST CRFR1 Receptors in Incubation of Fentanyl Seeking.

The time-dependent increase in cue-triggered opioid seeking, termed "incubation of opioid craving," is modeled in rodents by examining responding for opioid-associated cues after a period of forced abstinence. With opioid drugs, withdrawal symptoms may heighten cue reactivity by recruiting brain systems involved in both reward seeking and stress responses. Corticotropin releasing factor (CRF) in the bed nucleus of the stria terminalis (BNST) is a critical driver of stress-induced relapse to drug seeking. Here, we sought to determine whether BNST CRF receptor 1 (CRFR1) signaling drives incubation of opioid craving in opioid dependent and non-dependent rats. First, we tested whether BNST CRFR1 signaling drives incubation of opioid craving in rats with short-access fentanyl self-administration experience (2.5 µg/kg/infusion, 3 h/day for 10 days). On Day 1 of forced abstinence, we gave bilateral intra-BNST vehicle injections to all rats and measured lever responding for opioid cues in the absence of fentanyl infusions. On Day 30 of forced abstinence, we gave an identical test after bilateral intra-BNST injections of vehicle or CRFR1 receptor antagonist, R121919 (1 µg/0.3 µL/hemisphere). Vehicle treated rats showed greater responding for opioid associated cues on Day 30 relative to Day 1, and this incubation effect was prevented by intra-BNST R121919 on Day 30. Next, we incorporated an opioid-dependence procedure to investigate whether BNST CRFR1 signaling drives opioid cue-reactivity to a greater extent in opioid-dependent relative to non-dependent rats. We trained rats to self-administer fentanyl for 5 days before initiating the dependence phase and resuming daily fentanyl self-administration sessions for 10 days. We gave intra-BNST R121919 or vehicle injections before testing during acute (Day 5) or protracted (Day 30) withdrawal. During acute withdrawal, antagonizing BNST CRFR1 decreased the number of press bouts without affecting bout size or duration. These patterns of responding with R121919 treatment resulted in less fentanyl-associated conditioned reinforcement during test. Together, these findings suggest a role for BNST CRFR1 signaling in driving cue-reinforced opioid seeking after periods of forced abstinence.

Effects of Limited and Extended Pavlovian Training on Devaluation Sensitivity of Sign- and Goal-Tracking Rats.

Individual differences in Pavlovian approach predict differences in devaluation sensitivity. Recent studies indicate goal-tracking (GT) rats are sensitive to outcome devaluation while sign-tracking (ST) rats are not. With extended training in Pavlovian lever autoshaping (PLA), GT rats display more lever-directed behavior, typical of ST rats, suggesting they may become insensitive to devaluation with more Pavlovian training experience. Here, we use a within-subject satiety-induced outcome devaluation procedure to test devaluation sensitivity after limited and extended PLA training in GT and ST rats. We trained rats in PLA to determine GT and ST groups. Then, we sated rats on either the training pellets (devalued condition) or homecage chow (valued condition) prior to brief non-reinforced test sessions after limited (sessions 5/6) and extended (sessions 17/18) PLA training. GT rats decreased conditioned responding under devalued relative to valued conditions after both limited and extended training, demonstrating they are sensitive to satiety devaluation regardless of the amount of PLA training. While ST rats were insensitive to satiety devaluation after limited training, their lever directed behavior became devaluation sensitive after extended training. To determine whether sign-tracking rats also displayed sensitivity to illness-induced outcome devaluation after extended training, we trained a separate cohort of rats in extended PLA and devalued the outcome with lithium chloride injections after pellet consumption in the homecage. ST rats failed to decrease conditioned responding after illness-induced outcome devaluation, while Non-ST rats (GT and intermediates) were sensitive to illness-induced outcome devaluation after extended training. Together, our results confirm devaluation sensitivity is stable in GT rats across training and devaluation approaches. Extended training unmasks devaluation sensitivity in ST rats after satiety, but not illness-induced devaluation, suggesting ST rats respond appropriately by decreasing responding to cues during state-dependent but not inference-based devaluation. The differences in behavioral flexibility across tracking groups and devaluation paradigms have translational relevance for the understanding state- vs. inference-based reward devaluation as it pertains to drug addiction vulnerability.

Stability of individual differences in sucralose taste preference.

Outbred rats display variable preferences for bittersweet solutions, expressed as preference or avoidance of high concentrations of artificial sweeteners over water. This may reflect individual differences in appetitive/aversive conflict processing that may have predictive validity for disorders of motivation. Here we use a homecage two-bottle choice procedure to examine the test/retest stability and between-tastant consistency in sucralose preference to determine the reliability of bittersweet taste preference. Sucralose is a non-caloric artificial sweetener that is preferred by some rats and avoided by others. We sought to determine whether sucralose preference is consistent with preference of sucrose/quinine solutions that have known sweet and bitter taste qualities, respectively. We give fluid restricted rats 45-minutes homecage access to water and ascending concentrations of sucralose (SUCRA; 0.0025-10mM) or a compound solution of sucrose (116mM) + quinine (0.002-2mM) (SQ). We use a within-subject counterbalanced design (SUCRA or SQ testing) to determine preference of each bittersweet solution relative to water. We observed individual variability in preference for SUCRA and SQ, such that some rats preferred bittersweet solutions over water (preferring) while other rats preferred water over bittersweet solutions (avoiding). Within tastant, this preference remained stable across repeated testing. Between solutions, SUCRA preference scores correlated with SQ scores, suggesting consistent taste conflict processing for both bittersweet solutions. Population level analyses confirmed that preference generalizes across bittersweet solutions, and that rats' preferences for bittersweet solutions relative to water are stable over time. The test/retest and between-tastant reliability of this taste conflict screening procedure support the potential utility of this model for exploring individual variability in appetitive/aversive conflict processes mediating motivated behavior.

Manipulating the revision of reward value during the intertrial interval increases sign tracking and dopamine release.

Recent computational models of sign tracking (ST) and goal tracking (GT) have accounted for observations that dopamine (DA) is not necessary for all forms of learning and have provided a set of predictions to further their validity. Among these, a central prediction is that manipulating the intertrial interval (ITI) during autoshaping should change the relative ST-GT proportion as well as DA phasic responses. Here, we tested these predictions and found that lengthening the ITI increased ST, i.e., behavioral engagement with conditioned stimuli (CS) and cue-induced phasic DA release. Importantly, DA release was also present at the time of reward delivery, even after learning, and DA release was correlated with time spent in the food cup during the ITI. During conditioning with shorter ITIs, GT was prominent (i.e., engagement with food cup), and DA release responded to the CS while being absent at the time of reward delivery after learning. Hence, shorter ITIs restored the classical DA reward prediction error (RPE) pattern. These results validate the computational hypotheses, opening new perspectives on the understanding of individual differences in Pavlovian conditioning and DA signaling.

Cannabinoid receptor-1 signaling contributions to sign-tracking and conditioned reinforcement in rats.

RATIONALE: Endocannabinoids (eCBs) are critical gatekeepers of dopaminergic signaling, and disrupting cannabinoid receptor-1 (CB1) signaling alters DA dynamics to attenuate cue-motivated behaviors. Prior studies suggest that dopamine (DA) release plays a critical role in driving sign-tracking. OBJECTIVES: Here, we determine whether systemic injections of rimonabant, a CB1 receptor inverse agonist, during Pavlovian lever autoshaping impair the expression of sign-tracking. We next examine whether rimonabant blocks the reinforcing properties of the Pavlovian lever cue in a conditioned reinforcement test. METHODS: In Exp. 1, we trained rats in Pavlovian lever autoshaping prior to systemic rimonabant injections (0, 1, 3 mg/kg) during early and late Pavlovian lever autoshaping sessions. In Exp. 2, we trained rats in Pavlovian lever autoshaping prior to systemic rimonabant injections (0, 1 mg/kg) during a conditioned reinforcement test. RESULTS: Rimonabant dose-dependently decreased lever contact and probability, and increased sign-tracker's latency to approach the lever cue early in Pavlovian training. With extended training, many previously goal-tracking and intermediate rats shifted to lever approach, which remained dose-dependently sensitive to rimonabant. Rimonabant attenuated cue-evoked food cup approach early, but not late, in conditioning, and did not affect pellet retrieval or consumption. The inserted lever cue served as a robust conditioned reinforcer after Pavlovian lever autoshaping, and 1 mg/kg rimonabant blocked conditioned reinforcement. CONCLUSIONS: Together, our results suggest that CB1 signaling mediates two critical properties of incentive stimuli; their ability to attract (Exp. 1) and their ability to reinforce (Exp. 2) behavior.

Disconnection of basolateral amygdala and insular cortex disrupts conditioned approach in Pavlovian lever autoshaping.

Previously established individual differences in appetitive approach and devaluation sensitivity observed in goal- and sign-trackers may be attributed to differences in the acquisition, modification, or use of associative information in basolateral amygdala (BLA) pathways. Here, we sought to determine the extent to which communication of associative information between BLA and anterior portions of insular cortex (IC) supports ongoing Pavlovian conditioned approach behaviors in sign- and goal-tracking rats, in the absence of manipulations to outcome value. We hypothesized that the BLA mediates goal-, but not sign- tracking approach through interactions with the IC, a brain region involved in supporting flexible behavior. We first trained rats in Pavlovian lever autoshaping to determine their sign- or goal-tracking tendency. During alternating test sessions, we gave unilateral intracranial injections of vehicle or a cocktail of gamma-aminobutyric acid (GABA) receptor agonists, baclofen and muscimol, unilaterally into the BLA and contralaterally or ipsilaterally into the IC prior to reinforced lever autoshaping sessions. Consistent with our hypothesis we found that contralateral inactivation of BLA and IC increased the latency to approach the food cup and decreased the number of food cup contacts in goal-trackers. While contralateral inactivation of BLA and IC did not affect the total number of lever contacts in sign-trackers, this manipulation increased the latency to approach the lever. Ipsilateral inactivation of BLA and IC did not impact approach behaviors in Pavlovian lever autoshaping. These findings, contrary to our hypothesis, suggest that communication between BLA and IC maintains a representation of initially learned appetitive associations that commonly support the initiation of Pavlovian conditioned approach behavior regardless of whether it is directed at the cue or the location of reward delivery.

The Dopamine Prediction Error: Contributions to Associative Models of Reward Learning.

Phasic activity of midbrain dopamine neurons is currently thought to encapsulate the prediction-error signal described in Sutton and Barto's (1981) model-free reinforcement learning algorithm. This phasic signal is thought to contain information about the quantitative value of reward, which transfers to the reward-predictive cue after learning. This is argued to endow the reward-predictive cue with the value inherent in the reward, motivating behavior toward cues signaling the presence of reward. Yet theoretical and empirical research has implicated prediction-error signaling in learning that extends far beyond a transfer of quantitative value to a reward-predictive cue. Here, we review the research which demonstrates the complexity of how dopaminergic prediction errors facilitate learning. After briefly discussing the literature demonstrating that phasic dopaminergic signals can act in the manner described by Sutton and Barto (1981), we consider how these signals may also influence attentional processing across multiple attentional systems in distinct brain circuits. Then, we discuss how prediction errors encode and promote the development of context-specific associations between cues and rewards. Finally, we consider recent evidence that shows dopaminergic activity contains information about causal relationships between cues and rewards that reflect information garnered from rich associative models of the world that can be adapted in the absence of direct experience. In discussing this research we hope to support the expansion of how dopaminergic prediction errors are thought to contribute to the learning process beyond the traditional concept of transferring quantitative value.

Individual variability in behavioral flexibility predicts sign-tracking tendency.

Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS-) and second-order (SOCS+, SOCS-) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.

Intrinsic plasticity: an emerging player in addiction.

Exposure to drugs of abuse, such as cocaine, leads to plastic changes in the activity of brain circuits, and a prevailing view is that these changes play a part in drug addiction. Notably, there has been intense focus on drug-induced changes in synaptic excitability and much less attention on intrinsic excitability factors (that is, excitability factors that are remote from the synapse). Accumulating evidence now suggests that intrinsic factors such as K+ channels are not only altered by cocaine but may also contribute to the shaping of the addiction phenotype.