Effects of prior cocaine versus morphine or heroin self-administration on extinction learning driven by overexpectation versus omission of reward.

BACKGROUND: Addiction is characterized by an inability to stop using drugs, despite adverse consequences. One contributing factor to this compulsive drug taking could be the impact of drug use on the ability to extinguish drug seeking after changes in expected outcomes. Here, we compared effects of cocaine, morphine, and heroin self-administration on two forms of extinction learning: standard extinction driven by reward omission and extinction driven by reward overexpectation. METHODS: In experiment 1, we trained rats to self-administer cocaine, morphine, or sucrose for 3 hours per day (limited access). In experiment 2, we trained rats to self-administer heroin or sucrose for 12 hours per day (extended access). Three weeks later, we trained the rats to associate several cues with palatable food reward, after which we assessed extinction of the learned Pavlovian response, first by pairing two cues together in the overexpectation procedure and later by omitting the food reward. RESULTS: Rats trained under limited access conditions to self-administer sucrose or morphine demonstrated normal extinction in response to both overexpectation and reward omission, whereas cocaine-experienced rats or rats trained to self-administer heroin under extended access conditions exhibited normal extinction in response to reward omission but failed to show extinction in response to overexpectation. CONCLUSIONS: Here we show that cocaine and heroin can induce long-lasting deficits in the ability to extinguish reward seeking. These deficits were not observed in a standard extinction procedure but instead only affected extinction learning driven by a more complex phenomenon of overexpectation.

Neural estimates of imagined outcomes in the orbitofrontal cortex drive behavior and learning.

Imagination, defined as the ability to interpret reality in ways that diverge from past experience, is fundamental to adaptive behavior. This can be seen at a simple level in our capacity to predict novel outcomes in new situations. The ability to anticipate outcomes never before received can also influence learning if those imagined outcomes are not received. The orbitofrontal cortex is a key candidate for where the process of imagining likely outcomes occurs; however, its precise role in generating these estimates and applying them to learning remain open questions. Here we address these questions by showing that single-unit activity in the orbitofrontal cortex reflects novel outcome estimates. The strength of these neural correlates predicted both behavior and learning, learning that was abolished by temporally specific inhibition of orbitofrontal neurons. These results are consistent with the proposal that the orbitofrontal cortex is critical for integrating information to imagine future outcomes.

Normal Aging does Not Impair Orbitofrontal-Dependent Reinforcer Devaluation Effects.

Normal aging is associated with deficits in cognitive flexibility thought to depend on prefrontal regions such as the orbitofrontal cortex (OFC). Here, we used Pavlovian reinforcer devaluation to test whether normal aging might also affect the ability to use outcome expectancies to guide appropriate behavioral responding, which is also known to depend on the OFC. Both young and aged rats were trained to associate a 10-s conditioned stimulus (CS+) with delivery of a sucrose pellet. After training, half of the rats in each age group received the sucrose pellets paired with illness induced by LiCl injections; the remaining rats received sucrose and illness explicitly unpaired. Subsequently, responding to the CS+ was assessed in an extinction probe test. Although aged rats displayed lower responding levels overall, both young and aged rats conditioned to the CS+ and developed a conditioned taste aversion following reinforcer devaluation. Furthermore, during the extinction probe test, both young and aged rats spontaneously attenuated conditioned responding to the cue as a result of reinforcer devaluation. These data show that normal aging does not affect the ability to use expected outcome value to appropriately guide Pavlovian responding. This result indicates that deficits in cognitive flexibility are dissociable from other known functions of prefrontal - and particularly orbitofrontal - cortex.

The role of the nucleus accumbens in knowing when to respond.

While knowing what to expect is important, it is equally important to know when to expect it and to respond accordingly. This is apparent even in simple Pavlovian training situations in which animals learn to respond more strongly closer to reward delivery. Here we report that the nucleus accumbens core, an area well-positioned to represent information about the timing of impending rewards, plays a critical role in this timing function.

Neural correlates of variations in event processing during learning in central nucleus of amygdala.

Attention or variations in event processing help drive learning. Lesion studies have implicated the central nucleus of the amygdala (CeA) in this process, particularly when expected rewards are omitted. However, lesion studies cannot specify how information processing in CeA supports such learning. To address these questions, we recorded CeA neurons in rats performing a task in which rewards were delivered or omitted unexpectedly. We found that activity in CeA neurons increased selectively at the time of omission and declined again with learning. Increased firing correlated with CeA-inactivation sensitive measures of attention. Notably CeA neurons did not fire to the cues or in response to unexpected rewards. These results indicate that CeA contributes to learning in response to reward omission due to a specific role in signaling actual omission rather than a more general involvement in signaling expectancies, errors, or reward value.

Nucleus Accumbens Core and Shell are Necessary for Reinforcer Devaluation Effects on Pavlovian Conditioned Responding.

The nucleus accumbens (NA) has been hypothesized to be part of a circuit in which cue-evoked information about expected outcomes is mobilized to guide behavior. Here we tested this hypothesis using a Pavlovian reinforcer devaluation task, previously applied to assess outcome-guided behavior after damage to regions such as the orbitofrontal cortex and amygdala that send projections to NA. Rats with sham lesions or neurotoxic lesions of either the core or shell subdivision of NA were trained to associate a 10-s CS+ with delivery of three food pellets. After training, half of the rats in each lesion group received food paired with illness induced by LiCl injections; the remaining rats received food and illness unpaired. Subsequently, responding to the CS+ was assessed in an extinction probe test. Both sham and lesioned rats conditioned to the CS+ and formed a conditioned taste aversion. However only sham rats reduced their conditioned responding as a result of reinforcer devaluation; devalued rats with lesions of either core or shell showed levels of responding that were similar to lesioned, non-devalued rats. This impairment was not due to the loss of motivational salience conferred to the CS+ in lesioned rats as both groups responded similarly for the cue in conditioned reinforcement testing. These data suggest that NA core and shell are part of a circuit necessary for the use of cue-evoked information about expected outcomes to guide behavior.

Inactivation of the central but not the basolateral nucleus of the amygdala disrupts learning in response to overexpectation of reward.

The amygdala is critical for associating predictive cues with primary rewarding and aversive outcomes. This is particularly evident in tasks in which information about expected outcomes is required for normal responding. Here we used a pavlovian overexpectation task to test whether outcome signaling by amygdala might also be necessary for changing those representations in the face of unexpected outcomes. Rats were trained to associate several different cues with a food reward. After learning, two of the cues were presented together, in compound, followed by the same reward. Before each compound training session, rats received infusions of 2,3-dioxo-6-nitro-1,2,3,4-tetrahydrobenzo[f]quinoxaline-7-sulfonamide or saline into either the basolateral (ABL) or central nucleus (CeN) of amygdala. We found that infusions into CeN abolished the normal decline in responding to the compounded cue in a later probe test, whereas infusions into ABL had no effect. These results are inconsistent with the proposal that signaling of information about expected outcomes by ABL contributes to learning, at least in this setting, and instead implicate the CeN in this process, perhaps attributable to the hypothesized involvement of this area in attention and variations in stimulus processing.

The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes.

Humans and other animals change their behavior in response to unexpected outcomes. The orbitofrontal cortex (OFC) is implicated in such adaptive responding, based on evidence from reversal tasks. Yet these tasks confound using information about expected outcomes with learning when those expectations are violated. OFC is critical for the former function; here we show it is also critical for the latter. In a Pavlovian overexpectation task, inactivation of OFC prevented learning driven by unexpected outcomes, even when performance was assessed later. We propose this reflects a critical contribution of outcome signaling by OFC to encoding of reward prediction errors elsewhere. In accord with this proposal, we report that signaling of reward predictions by OFC neurons was related to signaling of prediction errors by dopamine neurons in ventral tegmental area (VTA). Furthermore, bilateral inactivation of VTA or contralateral inactivation of VTA and OFC disrupted learning driven by unexpected outcomes.