Error-Driven Learning: Dopamine Signals More Than Value-Based Errors.

The detection of a mismatch between our predictions and what actually happens allows us to learn from our errors. New research indicates that midbrain dopamine neurons encode multiple types of error signals and contribute to multiple forms of error-driven learning.

Unilateral inactivation of the basolateral amygdala attenuates context-induced renewal of Pavlovian-conditioned alcohol-seeking.

Environmental contexts associated with drug use promote craving in humans and drug-seeking in animals. We hypothesized that the basolateral amygdala (BLA) itself as well as serial connectivity between the BLA and nucleus accumbens core (NAC core) were required for context-induced renewal of Pavlovian-conditioned alcohol-seeking. Male Long-Evans rats were trained to discriminate between two conditioned stimuli (CS): a CS+ that was paired with ethanol (EtOH, 20%, v/v) delivery into a fluid port (0.2 mL/CS+, 3.2 mL per session) and a CS- that was not. Entries into the port during each CS were measured. Next, rats received extinction in a different context where both cues were presented without EtOH. At test, responding to the CS+ and CS- without EtOH was evaluated in the prior training context. Control subjects showed a selective increase in CS+ responding relative to extinction, indicative of renewal. This effect was blocked by pre-test, bilateral inactivation of the BLA using a solution of GABA receptor agonists (0.1 mm muscimol and 1.0 mm baclofen; M/B; 0.3 µL per side). Renewal was also attenuated following unilateral injections of M/B into the BLA, combined with either M/B, the dopamine D1 receptor antagonist SCH 23390 (0.6 µg per side) or saline infusion in the contralateral NAC core. Hence, unilateral BLA inactivation was sufficient to disrupt renewal, highlighting a critical role for functional activity in the BLA in enabling the reinstatement of alcohol-seeking driven by an alcohol context.

Downregulation of mu opioid receptor by RNA interference in the ventral tegmental area reduces ethanol consumption in mice.

Pharmacological and genetic studies have implicated the mu opioid receptor (MOR) in the regulation of ethanol intake in animal models and humans. Non-specific antagonists of opioid receptors have been shown to affect ethanol consumption when infused directly into the ventral tegmental area (VTA) of rats. However, administration of MOR-selective antagonists into the VTA has yielded mixed results. We used RNA interference (RNAi) to specifically decrease levels of MOR messenger RNA in the VTA of mice and examined the effect on ethanol consumption in a two-bottle choice paradigm. Mice were injected in the VTA with lentivirus expressing either a small hairpin RNA (shRNA) targeting MOR or a control shRNA. One week after virus injection, mice were examined for ethanol consumption in a two-bottle choice experiment with increasing concentrations of ethanol over the course of 1 month. Expression of an shRNA targeting MOR in the VTA led to a significant reduction in ethanol consumption. These results strengthen the hypothesis that MOR in the VTA is one of the key brain substrates mediating alcohol consumption. The RNAi combined with lentiviral delivery can be used successfully in brain to effect a sustained reduction in expression of specific genes for behavioral analysis.

The ethanol self-administration context as a reinstatement cue: acute effects of naltrexone.

A major difficulty in treating alcohol addiction is the high rate of relapse even after prolonged abstinence. Relapse can be triggered by several factors, including stress, re-exposure to the drug, conditioned discrete stimuli and exposure to the context in which alcohol consumption occurred. The present study investigated the role played by the environmental context on ethanol relapse using an extinction/reinstatement animal model: rats were trained to self-administer ethanol in a distinctive context, and extinction occurred in a setting that differed by visual, tactile and olfactory properties; reinstatement was tested by placing the animals into the ethanol-associated context in the absence of ethanol. We found that re-exposure to the ethanol-associated context significantly increased responses on the ethanol-paired lever. The increase in responding required the presence of the complete configuration of the multimodal context. The non-selective opioid receptor antagonist naltrexone (0.3 mg/kg) administered 20 min prior to the reinstatement test significantly attenuated context-induced reinstatement of lever press responding, compared with saline-treated subjects. These data indicate that the environmental context associated with ethanol availability influences ethanol-seeking behavior in the rat, and that endogenous opioids are involved in this process. Our findings are in accordance with clinical reports demonstrating naltrexone efficacy in the treatment of alcohol relapse in humans, and indicate that the context-induced reinstatement model described here may be useful to investigate the biological mechanisms underlying alcohol relapse.

Positive relationship between activity in a novel environment and operant ethanol self-administration in rats.

RATIONALE: The study of individual differences in drug addiction may have important implications both for understanding the etiology of addiction and for strategies for treatment. Activity of rodents in novel environments, presumably related to the novelty-seeking trait in humans, is the primary behavioral feature that is hypothesized to predict a predisposition for drug self-administration by rodents. OBJECTIVES: The aim of this study was to characterize the relationship between motor activity in a novel environment and operant ethanol self-administration using the sucrose-substitution procedure. METHODS: Male Long-Evans rats were exposed to a novel environment for 2 h, and the distance traversed, rearing, and defecation was recorded. After 3 days of forced exposure to ethanol the sucrose-substitution procedure began and lasted for 23 days. Following sucrose substitution the animals were maintained on a schedule of ethanol (10% v/v) self-administration with a fixed ratio 3 (FR3) for 15 days. RESULTS: The activity (distance traversed) in the novel environment was positively correlated with initial ethanol self-administration under the FR3 schedule ( r=0.87) but not with the number of inactive lever presses or active lever presses for ethanol or for sweetened ethanol solutions with lower ratios of response. In contrast, rearing was correlated positively only with the number of inactive lever presses for sucrose. CONCLUSIONS: Motor activity in a novel environment may be related to the acquisition of operant ethanol self-administration only when a given ratio of response is required.

In vivo extracellular recording of striatal neurons in the awake rat following unilateral 6-hydroxydopamine lesions.

The purpose of this study was to further understand the functional effects of dopaminergic input to the dorsal striatum and to compare the effects of dopaminergic lesions in awake and anesthetized animals. We examined the effects of unilateral 6-hydroxydopamine (6-OHDA) lesions of the ascending dopaminergic bundle on the firing properties of dorsal striatal neurons in the awake freely moving rat using chronically implanted microwire electrode arrays. We recorded extracellular activity of striatal neurons under baseline conditions and following the systemic injection of apomorphine in awake and anesthetized subjects. Firing rates were higher in the hemisphere ipsilateral to the 6-OHDA lesion compared to rates of neurons from the contralateral unlesioned hemisphere. Striatal firing rates from sham and no-surgery control rats were, in general, higher than those from the contralateral unlesioned striatum of experimental subjects. Apomorphine (0.05 mg/kg, sc) normalized the differences in firing rates in lesioned animals by increasing firing of neurons within the contralateral unlesioned side, while simultaneously decreasing firing of neurons within the ipsilateral lesioned side. Mean firing rates were substantially higher in awake animals than in subjects anesthetized with chloral hydrate, perhaps reflecting anesthesia-induced decreases in excitatory input to striatal neurons. Chloral hydrate anesthesia decreased firing rates of neurons in the lesioned, unlesioned, and control striata to a similar degree, although absolute firing rates of neurons from the 6-OHDA-lesioned striata remained elevated over all other groups. Unilateral 6-OHDA lesions also altered the pattern of spike output in the awake animal as indicated by an increase in the number of bursts per minute following dopaminergic deafferentation. This and other burst parameters were altered by apomorphine. Our findings show that effects of dopaminergic deafferentation can be measured in the awake behaving animal; this model should prove useful for testing the behavioral and functional effects of experimental manipulations designed to reduce or reverse the effects of dopaminergic cell loss. In addition, these results suggest that the contralateral changes in striatal function which occur in the unilateral dopaminergic lesion model should be considered when evaluating experimental results.

Neuronal and behavioral correlations in the medial prefrontal cortex and nucleus accumbens during cocaine self-administration by rats.

Up to 31 neurons per animal were simultaneously recorded from the medial prefrontal cortex and nucleus accumbens in 15 rats during i.v. cocaine self-administration sessions, using a multi-channel, single-unit recording technique. Alterations of neuronal activity (both excitatory and inhibitory) were found a few seconds before each lever press for cocaine infusion; we have called these pre-lever press neuronal activations "anticipatory responses". A detailed video analysis revealed that these neuronal firing alterations were associated with specific portions of the behavioral sequence performed before each lever press in both recording areas. Some of the simultaneously recorded neurons displayed similar firing patterns in relation to a given behavioral episode within the behavioral sequence (turning, raising head, etc.), while others fired at different times relative to each behavioral event. Cross-correlational analyses revealed inter-regional and intra-regional correlated firing patterns between pairs of simultaneously recorded medial prefrontal cortex and nucleus accumbens neurons. This correlated firing occurred in the neurons with and without anticipatory responses, although the incidence of correlations between anticipatory neuron pairs was much higher than that between non-anticipatory neuron pairs (18.4% vs 3.6%). Many correlated neuron pairs displayed a time lag in the peak of correlational activity that indicated a temporal sequence in correlated activity. In contradiction to our hypothesis, the temporal pattern of correlation reveals that there are more cases in which nucleus accumbens neurons fired ahead of medial prefrontal cortex neurons. The results suggest that multiple mesocorticolimbic neuronal circuits may code sequential steps during the behavioral sequence performed to obtain an infusion of cocaine. The observed correlated firing between the medial prefrontal cortex and the nucleus accumbens indicates that dynamic, coherent activity occurs within the mesocorticolimbic circuit. Because this circuit is hypothesized to drive drug-seeking behavior, we suggest that this correlated firing between the nucleus accumbens and the medial prefrontal cortex may participate in the control of cocaine self-administration. In addition, the finding that correlated activity within the nucleus accumbens more often precedes that of the medial prefrontal cortex suggests that the nucleus accumbens may play a prime role in the initiation of cocaine self-administration.

Neuronal spike activity in the nucleus accumbens of behaving rats during ethanol self-administration.

Many lines of evidence support the importance of the nucleus accumbens (NAC) for ethanol-reinforced behavior. The nature of the neuronal activity that occurs in this region during ethanol self-administration is not known. We recorded from ensembles of single-units primarily located within the shell of the NAC during operant responding for oral ethanol solutions by well-trained rats. Of 90 units recorded from seven sessions from seven rats, 41 (46%) did not exhibit significant changes in relation to the experimental events. Of the 49 units (54%) that did exhibit significant phasic changes, alterations in firing rate occurred in relation to the following experimental events: operant response (63%), tone stimulus (20%), and ethanol delivery (63%). In addition, changes in spike activity during the intervals between the three experimental events were noted in 33% of the units. Most units (55% of responsive units) responded to multiple experimental events. Thus different but overlapping populations of neurons in the NAC represent each event that occurs along the temporal dimension of a single trial performed to obtain ethanol reward. The data suggest that the NAC plays a crucial role in linking together conditioned and unconditioned internal and external stimuli with motor plans to allow for ethanol-seeking behavior to occur.

The reinforcing effects of ethanol are altered by the endogenous neurosteroid, allopregnanolone.

We examined the effect of systemic administration of the endogenously occurring progesterone metabolite, allopregnanolone, on oral self-administration of ethanol by male rats. Rats were trained to perform an operant response for presentation of 0.1 ml of a solution of 10% ethanol in water using the sucrose fading technique. After acquisition of stable lever-press responding on a fixed-ratio 4 schedule, subjects received subcutaneous injections of 1, 3, or 10 mg/kg of allopregnanolone, or vehicle, 20 min prior to the self-administration session. Pretreatment with 3 mg/kg, but not 1 or 10 mg/kg, increased the mean total number of lever press responses made to obtain ethanol, and therefore increased the mean total number of ethanol presentations. The number of responses and response rate were examined as a function of the number of "runs" within the 30-min session; a "run" was defined as a series of consecutive responses with an interresponse interval of <1 min. The increase in total responses after 3 mg/kg was due in part to an increased number of responses for the first run of the session, with no effect on response rates. However, the higher dose of 10 mg/kg decreased response rates within the first run. Thus, allopregnanolone alters ethanol-reinforced responding at concentrations lower than those that depress rates of responding. The effects of administration of the benzodiazepene, diazepam, were determined for comparison with those of the neurosteroid. The subcutaneous injection of 0.3, 1.0, or 3.0 mg/kg of diazepam did not produce any clear dose-dependent changes in measures of ethanol-reinforced operant responding, supporting the suggestion of differences in the contribution of the benzodiazepene and neurosteroid binding sites to GABA(A) receptor function. The results indicate that exogenous administration of allopregnanolone dose-dependently alters ethanol-reinforced operant responding, and suggest that this endogenously occurring neurosteroid could mediate some of the reinforcing effects of ethanol.

Comparison of mesocorticolimbic neuronal responses during cocaine and heroin self-administration in freely moving rats.

To compare neuronal activity within the mesocorticolimbic circuit during the self-administration of cocaine and heroin, multiple-channel single-unit recordings of spike activity within the rat medial prefrontal cortex (mPFC) and nucleus accumbens (NAc) were obtained during the consecutive self-administration of cocaine and heroin within the same session. The variety of neuronal responses observed before the lever press are termed anticipatory responses, and those observed after the lever press are called post-drug infusion responses. For the total of the 110 mPFC and 111 NAc neurons recorded, 30-50% of neurons, depending on the individual sessions, had no alteration in spike activity in relation to either cocaine or heroin self-administration. Among the neurons exhibiting significant neuronal responses during a self-administration session, only a small portion (16-25%) of neurons responded similarly under both reinforcement conditions; the majority of neurons (75-84%) responded differently to cocaine and heroin self-administration as revealed by variations in both anticipatory and/or post-drug infusion responses. A detailed video analysis of specific movements to obtain the self-administration of both drugs provided evidence against the possibility that locomotive differences contributed to the observed differences in anticipatory responses. The overall mean activity of neurons recorded in mPFC and NAc measured across the duration of the session segment for either cocaine or heroin self-administration also was different for some neurons under the two reinforcement conditions. This study provides direct evidence that, in mPFC and NAc, heterogeneous neuronal circuits mediate cocaine and heroin self-administration and that distinct, but overlapping, subpopulations of neurons in these areas become active during operant responding for different reinforcers.

Ethanol action on neural networks studied with multineuron recording in freely moving animals.

The advent of new chronic multineuron recording techniques for examining neural activity in behaving animals has initiated a new phase in the analysis of the neuronal mechanisms that underlie ethanol and other drug self-administration. The technique allows for the simultaneous recording of groups of individual neurons in one or more brain regions during ongoing behavior; therefore, the spatio-temporal patterns of neuronal activity during specific behavioral events can be determined. We have successfully applied this technique to rat models of cocaine and heroin self-administration. Recently, using rats, we have been able to record from neurons in areas of the mesocorticolimbic circuit during ethanol-reinforced operant responding. In this review, we describe the current and future application of this new behavioral neurophysiology to the investigation of the neurobiology of alcohol addiction.

Rapid decay of cocaine-induced behavioral sensitization of locomotor behavior.

Sprague Dawley rats received three daily intraperitoneal (i.p.) injections of saline or 15 mg/kg cocaine. Following an interval of 2, 5 or 8 days, the behavioral response of separate groups of rats to a challenge injection of cocaine (15 mg/kg) was tested in an open field. After repeated cocaine (15 mg/kg) injection, movement in both the vertical and horizontal plane was increased in cocaine-treated rats 2, but not 5 or 8, days after treatment as compared to saline-treated subjects. In addition, behavioral ratings along an ordinal scale designed to reflect increases in behavioral activation were increased in cocaine-treated rats 2, but not 5 or 8, days after treatment. These results stand in contrast to other reports demonstrating long-lasting neural and behavioral changes after similar treatment regimens. Taken together, the results suggest that a treatment regimen of 15 mg/kg per day of cocaine for 3 days produces behavioral sensitization of locomotor behavior; however, this cocaine-induced behavioral sensitization does not persist beyond a few (< 5) days after repeated cocaine treatment, using the current experimental parameters.

Neuronal responses in prefrontal cortex and nucleus accumbens during heroin self-administration in freely moving rats.

Chronic multi-channel single unit recordings of neuronal responses in prefrontal cortex (PFC) and nucleus accumbens (NAc) were made in 9 male Sprague Dawley rats to determine patterns of neuronal activity during heroin self-administration. Up to 32 neurons were recorded simultaneously in these two brain regions while rats lever pressed on a continuous reinforcement schedule for intravenous infusion of heroin (30 microg/kg/infusion). The variety of neuronal responses observed before and after each self-administered heroin infusion can be classified according to the following categories: (1) neurons that increased or (2) decreased their activity immediately before the lever press; (3) neurons that increased or (4) decreased their activity after the heroin infusion; and, (5) neurons that did not alter their activity either before or after the lever press for heroin infusion. The majority (69% in the PFC and 65% in the NAc) of neurons sampled fell into this last category of no change, indicating that a selected fraction becomes active during this specific task. In general, NAc neurons displayed more post-heroin responses than PFC neurons while the proportion of neurons showing responses before the lever press was similar in the mPFC and the NAc. This initial description of the responses of PFC and NAc neurons during heroin self-administration suggests that the neuronal circuit of the mesocorticolimbic system is involved in heroin self-administration. This circuit appears to contribute both to the initiation of drug-seeking behavior (pre-lever press phasic neuronal responses), as well as the action of heroin infusion itself (post-infusion phasic neuronal responses) by activation of different subsets of neurons.

Cocaine impairs acquisition of an autoshaped lever-touch response.

The effects of daily peripheral (IP) post-session injection of cocaine on the development of an autoshaped lever-touch response in rats were investigated. Male Sprague-Dawley rats received ten daily pairings of a retractable lever (conditioned stimulus; CS) and food delivery (unconditioned stimulus; UCS). Food delivery occurred if the subjects contacted the extended lever within 10 s, or, if the subjects failed to contact the lever, at the end of the 10-s stimulus interval. These contingencies resulted in increased lever-touch responses over 10 days of conditioning. Cocaine (5.6-19.0 mg/kg) impaired development of the lever-touch response, as compared to saline-treated control subjects. Because the injections were given immediately after each conditioning session, we suggest that cocaine affects the neural processes involved in consolidation. Three additional control experiments support this suggestion. The effect of cocaine on lever-touch acquisition was time-dependent as daily injection of cocaine (5.6 mg/kg) 3 h after each conditioning session did not affect lever-touch acquisition. In addition, the effect of cocaine was dependent upon the explicit pairing of lever extension (CS) and food delivery (UCS) as immediate post-session cocaine (5.6 mg/kg) administration did not alter responding when the presentation of both the CS and the UCS was uncorrelated. Cocaine (5.6 mg/kg) administered to subjects previously trained to a performance criterion did not affect lever-touch responding, indicating that cocaine administration (5.6 mg/kg) impairs the development, but not the maintenance, of autoshaped lever-touch responding. In contrast, the highest dose of cocaine tested, 19.0 mg/kg, did decrease lever-touch responding in well-trained subjects, indicating that post-session administration of higher doses of cocaine can produce aversive effects that may affect both the acquisition and maintenance of appetitively motivated behavior in the rat. The relative contributions of the instrumental and classical associations inherent in the autoshaping procedure were investigated by altering response contingencies. Rats showed no evidence of learning the lever-touch response when lever insertion and food delivery were positively correlated, and no explicit response contingency was present (classical conditioning); further, cocaine-treated subjects did not differ from saline-treated subjects. However, cocaine did impair lever-touch responding in the instrumental version of the task. Taken together, these results show that the post-session administration of cocaine can impair the acquisition of a multi-trial, multi-session appetitively motivated response.

[Leu]enkephalin enhances active avoidance conditioning in rats and mice.

The effects of intraperitoneal (IP) administration of the endogenous opioid peptide, [Leu]enkephalin (LE), on avoidance conditioning in rodents were investigated. At a dose of 30 micrograms/kg (IP), LE enhanced acquisition of a one-way step-through active avoidance response when administered 2 minutes before training to Swiss Webster mice. [Leu]enkephalin produced a U-shaped dose-response function because both lower and higher doses of LE did not affect avoidance responding. [Leu]enkephalin-induced enhancement of avoidance acquisition was also observed in Sprague-Dawley rats; the intraperitoneal injection of 10 micrograms/kg LE, administered 5 minutes before training, enhanced acquisition of a jump-up one-way active avoidance response. When administered to Sprague-Dawley rats immediately after training, LE (30 micrograms/kg IP) enhanced jump-up avoidance responding at test 24 hours after peptide injection. Previously, we found LE to impair acquisition in the same tasks in both rats and mice, also at microgram doses, and also in a U-shaped manner. Thus, LE can either enhance or impair learning within the same species and the same task; these findings are in agreement with recent theoretical proposals regarding the nature of compounds, such as LE, that modulate learning and memory.

Uptake and metabolism of [3H]-Leu-enkephalin following either its intraperitoneal or subcutaneous administration to mice.

The uptake and metabolism of 30 micrograms/kg [3H]-Leu-enkephalin ([3H]-LE) following either intraperitoneal (IP) or subcutaneous (SC) administration to Swiss Webster mice was examined. Uptake of [3H] was rapid, with peak levels of radioactivity in plasma observed at 5 or 10 min following IP or SC peptide injection, respectively. The majority (80-99% +/- 0.8) of plasma radioactivity at all postinjection plasma collection time points was in the form of tyrosine-containing enkephalin metabolites, indicating a substantial and rapid in vivo hydrolysis rate for exogenously administered LE. Leu-enkephalin is metabolized in vivo faster than previously reported in vitro in mouse plasma. However, despite this extensive hydrolysis, levels of intact LE remaining in plasma following its systemic administration are within or above endogenous LE plasma levels.

Cocaine and amphetamine facilitate retention of jump-up responding in rats.

The effects of cocaine and d-amphetamine administration on the acquisition of an automated jump-up active avoidance task were examined in two separate experiments. On days 1 and 2, male Sprague-Dawley rats received one escape-only training trial, followed immediately by the intraperitoneal injection of cocaine, amphetamine, or saline. On day 3, subjects received eight escape/avoidance trials. The posttraining administration of cocaine (2.75 and 5.55 mg/kg) and amphetamine (0.3 and 1.0 mg/kg) on days 1 and 2 facilitated jump-up avoidance performance on day 3. Importantly, both cocaine and amphetamine enhanced learning and memory under experimental conditions that allowed for drug-free training and testing.

Cocaine enhances one-way avoidance responding in mice.

We reported previously that posttraining cocaine injections enhance subsequent performance of an automated jump-up avoidance response and a trough avoidance response in rats. In the present study we examined the species generality of the cocaine enhancement by investigating the effects of posttraining cocaine injection on subsequent performance of a one-way active avoidance response in mice. Cocaine (30 mg/kg, IP) administered to mice immediately following completion of two escape-only trials on day 1 significantly enhanced avoidance response performance on day 2. Neither lidocaine nor cocaine methiodide, when administered in doses equimolar to the effective cocaine dose, altered performance on day 2. These data indicate that cocaine's enhancement of avoidance responding in mice probably is neither peripherally mediated nor attributable to its local anesthetic properties.

Cocaine enhances retention of avoidance conditioning in rats.

The effects of post-training cocaine administration were tested on retention of a one-way active avoidance task in rats. A 5.0 mg/kg IP dose of cocaine enhanced retention of the avoidance task, in three separate experiments, as indicated by an increase in the number of avoidances made when animals were tested 24 h after training, while both a lower (2.5 mg/kg) and a higher (7.5 mg/kg) cocaine dose had no effect. Lidocaine (4-8 mg/kg) administered post-training did not reliably affect retention in the same task. Cocaine's ability to enhance retention depended on the interval between training and drug injection such that only cocaine administered directly after training enhanced retention the following day. The results show that post-training cocaine administration enhances retention of an active avoidance task in rats, and that this effect is probably independent of the anesthetic properties of the drug.