Striatal dopamine release tracks the relationship between actions and their consequences.

The acquisition and performance of goal-directed actions has long been argued to depend on the integration of glutamatergic inputs to the posterior dorsomedial striatum (pDMS) under the modulatory influence of dopamine. Nevertheless, relatively little is known about the dynamics of striatal dopamine during goal-directed actions. To investigate this, we chronically recorded dopamine release in the pDMS as rats acquired two actions for distinct outcomes as these action-outcome associations were incremented and then subsequently degraded or reversed. We found that bilateral dopamine release scaled with action value, whereas the lateralized dopamine signal, i.e., the difference in dopamine release ipsilaterally and contralaterally to the direction of the goal-directed action, reflected the strength of the action-outcome association independently of changes in movement. Our results establish, therefore, that striatal dopamine activity during goal-directed action reflects both bilateral moment-to-moment changes in action value and the long-term action-outcome association.

Stimulus control of habits: Evidence for both stimulus specificity and devaluation insensitivity in a dual-response task.

Goal-directed and habitual actions are clearly defined by their associative relations. Whereas goal-directed control can be confirmed via tests of outcome devaluation and contingency-degradation sensitivity, a comparable criterion for positively detecting habits has not been established. To confirm habitual responding, a test of control by the stimulus-response association is required while also ruling out goal-directed control. Here we describe an approach to developing such a test in rats using two discriminative stimuli that set the occasion for two different responses that then earn the same outcome. Performance was insensitive to outcome devaluation and showed stimulus-response specificity, indicative of stimulus-controlled behavior. The reliance of stimulus-response associations was further supported by a lack of sensitivity during the single extinction test session used here. These results demonstrate that two concurrently trained responses can come under habitual control when they share a common outcome. By reducing the ability of one stimulus to signal its corresponding response-outcome association, we found evidence for goal-directed control that can be dissociated from habits. Overall, these experiments provide evidence that tests assessing specific stimulus-response associations can be used to investigate habits.

Right external globus pallidus changes are associated with altered causal awareness in youth with depression.

Cognitive impairment is a functionally disabling feature of depression contributing to maladaptive decision-making, a loss of behavioral control and an increased disease burden. The ability to calculate the causal efficacy of ones actions in achieving specific goals is critical to normal decision-making and, in this study, we combined voxel-based morphometry (VBM), shape analysis and diffusion tensor tractography to investigate the relationship between cortical-basal ganglia structural integrity and such causal awareness in 43 young subjects with depression and 21 demographically similar healthy controls. Volumetric analysis determined a relationship between right pallidal size and sensitivity to the causal status of specific actions. More specifically, shape analysis identified dorsolateral surface vertices where an inward location was correlated with reduced levels of causal awareness. Probabilistic tractography revealed that affected parts of the pallidum were primarily connected with the striatum, dorsal thalamus and hippocampus. VBM did not reveal any whole-brain gray matter regions that correlated with causal awareness. We conclude that volumetric reduction within the indirect pathway involving the right dorsolateral pallidum is associated with reduced awareness of the causal efficacy of goal-directed actions in young depressed individuals. This causal awareness task allows for the identification of a functionally and biologically relevant subgroup to which more targeted cognitive interventions could be applied, potentially enhancing the long-term outcomes for these individuals.

Disruption of endogenous opioid activity during instrumental learning enhances habit acquisition.

Considerable evidence suggests that in instrumental conditioning rats learn the relationship between actions and their consequences, or outcomes. Such goal-directed actions are sensitive to changes in outcome value. The present study assessed the role of the endogenous opioid system in goal-directed reward learning. In two experiments, rats were trained to lever press for food pellets either under vehicle or naloxone-induced opioid receptor blockade. Specific satiety procedures were used for outcome devaluation, and the effect of this devaluation on instrumental responding was then tested in extinction. In Experiment 1 outcome devaluation resulted in a reduction in lever pressing in rats that were trained after vehicle injections, indicating that actions in these rats were goal-directed. In contrast, actions in rats trained under naloxone were insensitive to outcome devaluation when tested off drug, suggesting that lever pressing had become habitual in these rats. Interestingly, in Experiment 2 naloxone-induced habitual behavior was shown to be specific to the context in which the training occurred under naloxone; rats showed normal sensitivity to outcome devaluation when tested in an alternate vehicle-trained context. Additionally, in Experiment 2 we found that the acute administration of naloxone on test had no effect in itself, indicating that opioid receptor-related processes contribute to the acquisition of goal-directed actions and not to their general performance. These data suggest that an intact endogenous opioid system is necessary for normal goal-directed learning and more importantly, reveal that a compromised endogenous opioid system during learning enhances the habitual control of actions.

Distinct opioid circuits determine the palatability and the desirability of rewarding events.

It generally is assumed that a common neural substrate mediates both the palatability and the reward value of nutritive events. However, recent evidence suggests this assumption may not be true. Whereas opioid circuitry in both the nucleus accumbens and ventral pallidum has been reported to mediate taste-reactivity responses to palatable events, the assignment of reward or inventive value to goal-directed actions has been found to involve the basolateral amygdala. Here we found that, in rats, the neural processes mediating palatability and incentive value are indeed dissociable. Naloxone infused into either the ventral pallidum or nucleus accumbens shell blocked the increase in sucrose palatability induced by an increase in food deprivation without affecting the performance of sucrose-related actions. Conversely, naloxone infused into the basolateral amygdala blocked food deprivation-induced changes in sucrose-related actions without affecting sucrose palatability. This double dissociation of opioid-mediated changes in palatability and incentive value suggests that the role of endogenous opioids in reward processing does not depend on a single neural circuit. Rather, changes in palatability and in the incentive value assigned to rewarding events seem to be mediated by distinct neural processes.

Sensorimotor gating abnormalities in young males with fragile X syndrome and Fmr1-knockout mice.

Fragile X syndrome (FXS) is the most common single gene (FMR1) disorder affecting cognitive and behavioral function in humans. This syndrome is characterized by a cluster of abnormalities including lower IQ, attention deficits, impairments in adaptive behavior and increased incidence of autism. Here, we show that young males with FXS have profound deficits in prepulse inhibition (PPI), a basic marker of sensorimotor gating that has been extensively studied in rodents. Importantly, the magnitude of the PPI impairments in the fragile X children predicted the severity of their IQ, attention, adaptive behavior and autistic phenotypes. Additionally, these measures were highly correlated with each other, suggesting that a shared mechanism underlies this complex phenotypic cluster. Studies in Fmr1-knockout mice also revealed sensorimotor gating and learning abnormalities. However, PPI and learning were enhanced rather than reduced in the mutants. Therefore, these data show that mutations of the FMR1 gene impact equivalent processes in both humans and mice. However, since these phenotypic changes are opposite in direction, they also suggest that murine compensatory mechanisms following loss of FMR1 function differ from those in humans.

Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in rats.

In two experiments the involvement of the nucleus accumbens in instrumental conditioning was investigated using rats as subjects. In experiment 1, extensive bilateral cytotoxic lesions of the nucleus accumbens mildly suppressed instrumental responding reinforced with food, but had no detectable effect on the sensitivity of the rats' performance either to outcome devaluation or to degradation of the instrumental contingency. In experiment 2, restricted accumbens lesions reliably attenuated the excitatory effect of systemically administered d-amphetamine on lever pressing for a conditioned reinforcer, and completely abolished Pavlovian-instrumental transfer. Taken together these results give a picture of the involvement of the rat nucleus accumbens in instrumental conditioning. They support the widely held theory that the nucleus accumbens mediates the excitatory effects of appetitively conditioned Pavlovian signals on instrumental performance and refute the hypothesis that the nucleus accumbens is part of the neural circuitry by which incentive value is attached to the representations of instrumental outcomes.

The role of the nucleus accumbens in instrumental conditioning: Evidence of a functional dissociation between accumbens core and shell.

In three experiments we examined the effect of bilateral excitotoxic lesions of the nucleus accumbens core or shell subregions on instrumental performance, outcome devaluation, degradation of the instrumental contingency, Pavlovian conditioning, and Pavlovian-instrumental transfer. Rats were food deprived and trained to press two levers, one delivering food pellets and the other a sucrose solution. All animals acquired the lever-press response although the rate of acquisition and overall response rates in core-lesioned animals were depressed relative to that in the shell- or sham-lesioned animals. Furthermore, in shell- and sham-lesioned rats, post-training devaluation of one of the two outcomes using a specific satiety procedure produced a selective reduction in performance on the lever that, in training, delivered the prefed outcome. In contrast, the core-lesioned rats failed to show a selective devaluation effect and reduced responding on both levers. Subsequent tests revealed that these effects of core lesions were not caused by an impairment in their ability to recall the devalued outcome, to discriminate the two outcomes, or to encode the instrumental action-outcome contingencies to which they were exposed. Additionally, the core lesions did not have any marked effect on Pavlovian conditioning or on Pavlovian-instrumental transfer. Importantly, although shell-lesioned rats showed no deficit in any test of instrumental conditioning or in Pavlovian conditioning, they failed to show any positive transfer in the Pavlovian-instrumental transfer test. This double dissociation suggests that nucleus accumbens core and shell differentially mediate the impact of instrumental and Pavlovian incentive processes, respectively, on instrumental performance.

The role of the hippocampus in instrumental conditioning.

Considerable evidence suggests that, in instrumental conditioning, rats can encode both the specific action-outcome associations to which they are exposed and the degree to which an action is causal in producing its associated outcome. Three experiments assessed the involvement of the hippocampus in encoding these aspects of instrumental learning. In each study, rats with electrolytic lesions of the dorsal hippocampus and sham-lesioned controls were trained while hungry to press two levers, each of which delivered a unique food outcome. Experiments 1A and 1B used an outcome devaluation procedure to assess the effects of the lesion on encoding the action-outcome relationship. After training, one of the two outcomes was devalued using a specific satiety procedure, after which performance on the two levers was assessed in a choice extinction test. The lesion had no detectable effect on either the acquisition of instrumental performance or on the rats' sensitivity to outcome devaluation; lesion and sham groups both reduced responding on the lever associated with the devalued outcome compared with the other lever. In experiment 2, the sensitivity of hippocampal rats to the causal efficacy of their actions was assessed by selectively degrading the contingency between one of the actions and its associated outcome. Whereas sham rats selectively reduced performance on the lever for which the action-outcome contingency had been degraded, hippocampal rats did not. These results suggest that, in instrumental conditioning, lesions of the dorsal hippocampus selectively impair the ability of rats to represent the causal relationship between an action and its consequences.

The effect of lesions of the insular cortex on instrumental conditioning: evidence for a role in incentive memory.

In three experiments, we assessed the effect of lesions aimed at the gustatory region of the insular cortex on instrumental conditioning in rats. In experiment 1, the lesion had no effect on the acquisition of either lever pressing or chain pulling in food-deprived rats whether these actions earned food pellets or a maltodextrin solution. The lesion did, however, attenuate the impact of outcome devaluation, induced by sensory-specific satiety, on instrumental performance but only when assessed in an extinction test. This effect was not secondary to an impairment in instrumental learning; in experiment 2, no evidence was found to suggest that the lesioned rats differed from shams in their ability to encode the specific action-outcome contingencies to which they were exposed during training. In experiment 3, however, lesioned rats were found to be insensitive to the impact of an incentive learning treatment conducted when they were undeprived; although, again, this deficit was confined to a test conducted in extinction. These results are consistent with the view that, in instrumental conditioning, the gustatory region of the insular cortex is involved in encoding the taste of food outcomes in memory and, hence, in encoding the incentive value assigned to these outcomes on the basis of prevailing motivational conditions.

Goal-directed instrumental action: contingency and incentive learning and their cortical substrates.

Instrumental behaviour is controlled by two systems: a stimulus-response habit mechanism and a goal-directed process that involves two forms of learning. The first is learning about the instrumental contingency between the response and reward, whereas the second consists of the acquisition of incentive value by the reward. Evidence for contingency learning comes from studies of reward devaluation and from demonstrations that instrumental performance is sensitive not only the probability of contiguous reward but also to the probability of unpaired rewards. The process of incentive learning is evident in the acquisition of control over performance by primary motivational states. Preliminary lesion studies of the rat suggest that the prelimbic area of prefrontal cortex plays a role in the contingency learning, whereas the incentive learning for food rewards involves the insular cortex.

Effect of the 5-HT1A agonist, 8-OH-DPAT on instrumental performance in rats.

These experiments assessed whether reported increases in food consumption and food-reinforced instrumental performance in undeprived rats by the 5-HT1A agonist 8-hydroxy-2(di-n-propylamino)tetralin (8-OH-DPAT) are due to an increment in the incentive value of foods. Against this hypothesis, we found that when undeprived rats were trained to lever press for the food pellets and then allowed to consume the pellets under 8-OH-DPAT, this reexposure decreased subsequent instrumental extinction performance regardless of test drug condition relative to reexposure under vehicle. Although both food consumption and reinforced lever press performance were incremented, 8-OH-DPAT was found generally to reduce instrumental extinction performance and lever pressing during a period when the reinforcer was delivered non-contingently. Rats injected with 8-OH-DPAT were, however, more sensitive to delay of reinforcement, and increased their lever press performance at a 3-s delay but decreased performance at 6-s and 12-s delays relative to animals injected with vehicle. These results are consistent with the hypothesis that 8-OH-DPAT modifies arousal processes in a manner similar to mild stress, thereby acting both to elevate rewarded instrumental performance and to increase sensitivity to the effects of non-reward.

Differential effects of escapable and inescapable footshock on hippocampal theta activity.

Three groups of rats were exposed to escapable shock, inescapable shock, or no shock using procedures known to induce learned helplessness effects. Twenty-four hours later, hippocampal electroencephalograms were recorded from freely moving subjects. A 5-s probe shock was delivered after 15 min, and recording continued. Frequency analyses revealed no differences between the pretreatment groups during the first recording segment. Immediately after the probe shock, however, trains of immobility-related theta activity were observed in both the escapable-and no-shock groups. No such activity was observed in the inescapable-shock group. Because immobility followed the probe shock in all groups, this relative impairment was not due to differential motor activity. These results offer in vivo support for in vitro findings suggesting that hippocampal activity is sensitive to event contingencies and is involved in stress-induced learning deficits.

Environment-specific conditioning produced by electrical stimulation of the lateral hypothalamus.

Rats received noncontingent electrical stimulation of the lateral hypothalamus on one side of a place preference apparatus and no stimulation on the other side. Subsequently, when allowed access to both sides, the rats spent more time on the side associated with stimulation. This change in preference was only found in rats receiving stimulation in the side least preferred prior to conditioning trials. It was further shown that the place preference conditioning procedure produces increased locomotor activity. Thus, the place preference obtained was not an artifact produced by a conditioned freezing response. These data suggest that both the reinforcing and activating effects of lateral hypothalamic stimulation may be conditioned to a specific environment. Some methodological problems of the place preference paradigm are discussed.

Controllability of prestimulation of the medial prefrontal cortex determines the facilitation of self-stimulation and kindled seizures.

Electrical stimulation of the medial prefrontal cortex (MPC) was administered according to the triadic design typically used to demonstrate learned helplessness. Three groups received either controllable, uncontrollable or no stimulation during the pretreatment phase. The effects of this pretreatment on the acquisition of self-stimulation at the same electrode site were investigated in the second phase of the experiment. Relative to unstimulated controls, both controllable and uncontrollable prestimulation facilitated the acquisition of self-stimulation and produced higher self-stimulation rates. In addition, compared with controllable stimulation, pretreatment with uncontrollable stimulation produced a greater facilitation in self-stimulation rate. The unambiguous demonstration of a behavioural facilitation produced by pretreatment with uncontrollable stimulation is, effectively, the inverse of the typical learned helplessness finding. It was also found, in the second phase of the experiment, that 6 of the 7 rats previously exposed to uncontrollable stimulation developed full class 5 seizures. No behavioural evidence of kindling was seen in any of the other rats or during the prestimulation procedure. These data are interpreted in terms of kindling and stress effects both proximal and distal to the site of stimulation.

Footshock stress facilitates self-stimulation of the medial prefrontal cortex but not the lateral hypothalamus in the rat.

The effects of stress on self-stimulation were investigated by exposing rats to either controllable, uncontrollable or no footshock. Both controllable and uncontrollable footshock increased medial prefrontal cortex self-stimulation rates immediately as well as 24 h following treatment. Controllable footshock produced a greater enhancement than uncontrollable footshock. In contrast, self-stimulation of the lateral hypothalamus was unaffected by either footshock treatment. These results are interpreted with reference to the neurochemical response of the mesocortical dopaminergic system to acute stress.

Effects of ethanol and tertiary butanol on blood glucose levels and body temperature of rats.

The mechanisms of ethanol's hyperglycemic and hypothermic effects were investigated by comparing the effects of ethanol with those of tertiary butanol. Tertiary butanol is an intoxicant like ethanol, but unlike ethanol it is only minimally metabolized. Consequently, tertiary butanol does not produce appreciable amounts of active metabolites or energy. Tertiary butanol exerts its neural effects primarily by directly altering the physico-chemical properties of nerve cell membranes. It was found that ethanol and tertiary butanol produce hyperglycemic and hypothermic effects whose magnitude and time course are nearly identical. These data suggest that the hyperglycemic and hypothermic effects of ethanol represent a primary physico-chemical effect on nerve cell membranes and are not secondary to its energy content or metabolites.