Lesions to the mediodorsal thalamus, but not orbitofrontal cortex, enhance volatility beliefs linked to paranoia.

Beliefs-attitudes toward some state of the environment-guide action selection and should be robust to variability but sensitive to meaningful change. Beliefs about volatility (expectation of change) are associated with paranoia in humans, but the brain regions responsible for volatility beliefs remain unknown. The orbitofrontal cortex (OFC) is central to adaptive behavior, whereas the magnocellular mediodorsal thalamus (MDmc) is essential for arbitrating between perceptions and action policies. We assessed belief updating in a three-choice probabilistic reversal learning task following excitotoxic lesions of the MDmc (n = 3) or OFC (n = 3) and compared performance with that of unoperated monkeys (n = 14). Computational analyses indicated a double dissociation: MDmc, but not OFC, lesions were associated with erratic switching behavior and heightened volatility belief (as in paranoia in humans), whereas OFC, but not MDmc, lesions were associated with increased lose-stay behavior and reward learning rates. Given the consilience across species and models, these results have implications for understanding paranoia.

Sex differences in oxycodone-taking behaviors are linked to disruptions in reward-guided, decision-making functions.

Problematic opioid use that emerges in a subset of individuals may be due to pre-existing disruptions in the biobehavioral mechanisms that regulate drug use. The identity of these mechanisms is not known, but emerging evidence suggests that suboptimal decision-making that is observable prior to drug use may contribute to the pathology of addiction and, notably, serve as a powerful phenotype for interrogating biologically based differences in opiate-taking behaviors. The current study investigated the relationship between decision-making phenotypes and opioid-taking behaviors in male and female Long Evans rats. Adaptive decision-making processes were assessed using a probabilistic reversal-learning task and oxycodone- (or vehicle, as a control) taking behaviors assessed for 32 days using a saccharin fading procedure that promoted dynamic intake of oxycodone. Tests of motivation, extinction, and reinstatement were also performed. Computational analyses of decision-making and opioid-taking behaviors revealed that attenuated reward-guided decision-making was associated with greater self-administration of oxycodone and addiction-relevant behaviors. Moreover, pre-existing impairments in reward-guided decision-making observed in female rats was associated with greater oxycodone use and addiction-relevant behaviors when compared to males. These results provide new insights into the biobehavioral mechanisms that regulate opiate-taking behaviors and offer a novel phenotypic approach for interrogating sex differences in addiction susceptibility and opioid use disorders.

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related.

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

Examining sex differences in responses to footshock stress and the role of the metabotropic glutamate receptor 5: an [18F]FPEB and positron emission tomography study in rats.

Clinical investigations suggest involvement of the metabotropic glutamate receptor 5 (mGluR5) in the pathophysiology of fear learning that underlies trauma-related disorders. Here, we utilized a 4-day fear learning paradigm combined with positron emission tomography (PET) to examine the relationship between mGluR5 availability and differences in the response of rats to repeated footshock exposure (FE). Specifically, on day 1, male (n = 16) and female (n = 12) rats received 15 footshocks and were compared with control rats who did not receive footshocks (n = 7 male; n = 4 female). FE rats were classified as low responders (LR) or high responders (HR) based on freezing to the context the following day (day 2). PET with [18F]FPEB was used to calculate regional mGluR5 binding potential (BPND) at two timepoints: prior to FE (i.e., baseline), and post-behavioral testing. Additionally, we used an unbiased proteomics approach to assess group and sex differences in prefrontal cortex (PFC) protein expression. Post-behavioral testing we observed decreased BPND in LR females, but increased BPND in HR males relative to baseline. Further, individuals displaying the greatest freezing during the FE context memory test had the largest increases in PFC BPND. Males and females displayed unique post-test molecular profiles: in males, the greatest differences were between FE and CON, including upregulation of mGluR5 and related molecular networks in FE, whereas the greatest differences among females were between the LR and HR groups. These findings suggest greater mGluR5 availability increases following footshock exposure may be related to greater contextual fear memory. Results additionally reveal sex differences in the molecular response to footshock, including differential involvement of mGluR5-related molecular networks.

Adolescent reinforcement-learning trajectories predict cocaine-taking behaviors in adult male and female rats.

The anatomical, structural, and functional adaptations that occur in the brain during adolescence are thought to facilitate improvements in decision-making functions that are known to occur during this stage of development. The mechanisms that underlie these neural adaptations are not known, but deviations in developmental trajectories have been proposed to contribute to the emergence of mental illness, including addiction. Direct evidence supporting this hypothesis, however, has been limited. Here, we used a recently developed reversal-learning protocol to investigate the predictive relationship between adolescent decision-making trajectories and cocaine-taking behaviors in adulthood. Decision-making functions in the reversal-learning task were assessed throughout adolescence and into adulthood in male and female Long-Evans rats. Trial-by-trial choice data was fitted with a reinforcement-learning model to quantify the degree to which choice behavior of individual rats was influenced by rewarded (e.g., ∆+ parameter) and unrewarded (e.g., ∆0 parameter) outcomes. We report that reversal-learning performance improved during adolescence and that this was due to an increase in value updating for rewarded outcomes (e.g., ∆+ parameter). Furthermore, the rate of change in the ∆+ parameter predicted individual differences in the ∆+ parameter and, notably, cocaine-taking behaviors in adulthood: Rats that had a shallower adolescent trajectory were found to have a lower ∆+ parameter and greater cocaine self-administration in adulthood. These data indicate that adolescent development plays a critical role in drug use susceptibility. Future studies aimed at understanding the neurobiological mechanisms that underlie these age-related changes in decision-making could provide new insights into the biobehavioral mechanisms mediating addiction susceptibility.

Imaging the fetal nonhuman primate brain with SV2A positron emission tomography (PET).

PURPOSE: Exploring synaptic density changes during brain growth is crucial to understanding brain development. Previous studies in nonhuman primates report a rapid increase in synapse number between the late gestational period and the early neonatal period, such that synaptic density approaches adult levels by birth. Prenatal synaptic development may have an enduring impact on postnatal brain development, but precisely how synaptic density changes in utero are unknown because current methods to quantify synaptic density are invasive and require post-mortem brain tissue. METHODS: We used synaptic vesicle glycoprotein 2A (SV2A) positron emission tomography (PET) radioligands [11C]UCB-J and [18F]Syn-VesT-1 to conduct the first assessment of synaptic density in the developing fetal brain in gravid rhesus monkeys. Eight pregnant monkeys were scanned twice during the third trimester at two imaging sites. Fetal post-mortem samples were collected near term in a subset of subjects to quantify SV2A density by Western blot. RESULTS: Image-derived fetal brain SV2A measures increased during the third trimester. SV2A concentrations were greater in subcortical regions than in cortical regions at both gestational ages. Near term, SV2A density was higher in primary motor and visual areas than respective associative regions. Post-mortem quantification of SV2A density was significantly correlated with regional SV2A PET measures. CONCLUSION: While further study is needed to determine the exact relationship of SV2A and synaptic density, the imaging paradigm developed in the current study allows for the effective in vivo study of SV2A development in the fetal brain.

MicroPET evidence for a hypersensitive neuroinflammatory profile of gp120 mouse model of HIV.

Despite increased survivability for people living with HIV (PLWH), HIV-related cognitive deficits persist. Determining biological mechanism(s) underlying abnormalities is critical to minimize the long-term impact of HIV. Positron emission tomography (PET) studies reveal that PLWH exhibit elevated neuroinflammation, potentially contributing to these problems. PLWH are hypersensitive to environmental insults that drive elevated inflammatory profiles. Gp120 is an envelope glycoprotein exposed on the surface of the HIV envelope which enables HIV entry into a cell contributing to HIV-related neurotoxicity. In vivo evidence for mice overexpressing gp120 (transgenic) mice exhibiting neuroinflammation remains unclear. Here, we conducted microPET imaging in gp120 transgenic and wildtype mice, using the radiotracer [(18)F]FEPPA (binds to the translocator protein expressed by activated microglial serving as a neuroinflammatory marker). Imaging was performed at baseline and 24 h after lipopolysaccharide (LPS; 5 mg/kg) treatment (endotoxin that triggers an immune response). Gp120 transgenic mice exhibited elevated [(18F)]FEPPA in response to LPS vs. wildtype mice throughout the brain including dorsal and ventral striata, hypothalamus, and hippocampus. Gp120 transgenic mice are hypersensitive to environmental inflammatory insults, consistent with PLWH, measurable in vivo. It remains to-be-determined whether this heightened sensitivity is connected to the behavioral abnormalities of these mice or sensitive to any treatments.

The effects of fatty acid amide hydrolase inhibition and monoacylglycerol lipase inhibition on habit formation in mice.

Emerging data indicate that endocannabinoid signaling is critical to the formation of habitual behavior. Previous work demonstrated that antagonism of cannabinoid receptor type 1 (CB1R) with AM251 during operant training impairs habit formation, but it is not known if this behavioral effect is specific to disrupted signaling of the endocannabinoid ligands anandamide or 2-arachidonoyl glycerol (2-AG). Here, we used selective pharmacological compounds during operant training to determine the impact of fatty acid amide hydrolase (FAAH) inhibition to increase anandamide (and other n-acylethanolamines) or monoacylglycerol lipase (MAGL) inhibition to increase 2-AG levels on the formation of habitual behaviors in mice using a food-reinforced contingency degradation procedure. We found, contrary to our hypothesis, that inhibition of FAAH and of MAGL disrupted the formation of habits. Next, AM251 was administered during training to verify that impaired habit formation could be assessed using contingency degradation. AM251-exposed mice responded at lower rates during training and at higher rates in the test. To understand the inconsistency with published data, we performed a proof-of-principle dose-response experiment to compare AM251 in our vehicle-solution to the published vehicle-suspension on response rates. We found consistent reductions in response rate with increasing doses of AM251 in solution and an inconsistent dose-response relationship with AM251 in suspension. Together, our data suggest that further characterization of the role of CB1R signaling in the formation of habitual responding is warranted and that augmenting endocannabinoids may have clinical utility for prophylactically preventing aberrant habit formation such as that hypothesized to occur in substance use disorders.

Reinforcement learning detuned in addiction: integrative and translational approaches.

Suboptimal decision-making strategies have been proposed to contribute to the pathophysiology of addiction. Decision-making, however, arises from a collection of computational components that can independently influence behavior. Disruptions in these different components can lead to decision-making deficits that appear similar behaviorally, but differ at the computational, and likely the neurobiological, level. Here, we discuss recent studies that have used computational approaches to investigate the decision-making processes underlying addiction. Studies in animal models have found that value updating following positive, but not negative, outcomes is predictive of drug use, whereas value updating following negative, but not positive, outcomes is disrupted following drug self-administration. We contextualize these findings with studies on the circuit and biological mechanisms of decision-making to develop a framework for revealing the biobehavioral mechanisms of addiction.

Paranoia and belief updating during the COVID-19 crisis.

The COVID-19 pandemic has made the world seem less predictable. Such crises can lead people to feel that others are a threat. Here, we show that the initial phase of the pandemic in 2020 increased individuals' paranoia and made their belief updating more erratic. A proactive lockdown made people's belief updating less capricious. However, state-mandated mask-wearing increased paranoia and induced more erratic behaviour. This was most evident in states where adherence to mask-wearing rules was poor but where rule following is typically more common. Computational analyses of participant behaviour suggested that people with higher paranoia expected the task to be more unstable. People who were more paranoid endorsed conspiracies about mask-wearing and potential vaccines and the QAnon conspiracy theories. These beliefs were associated with erratic task behaviour and changed priors. Taken together, we found that real-world uncertainty increases paranoia and influences laboratory task behaviour.

Unlocking the reinforcement-learning circuits of the orbitofrontal cortex.

Neuroimaging studies have consistently identified the orbitofrontal cortex (OFC) as being affected in individuals with neuropsychiatric disorders. OFC dysfunction has been proposed to be a key mechanism by which decision-making impairments emerge in diverse clinical populations, and recent studies employing computational approaches have revealed that distinct reinforcement-learning mechanisms of decision-making differ among diagnoses. In this perspective, we propose that these computational differences may be linked to select OFC circuits and present our recent work that has used a neurocomputational approach to understand the biobehavioral mechanisms of addiction pathology in rodent models. We describe how combining translationally analogous behavioral paradigms with reinforcement-learning algorithms and sophisticated neuroscience techniques in animals can provide critical insights into OFC pathology in biobehavioral disorders. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Paranoia and belief updating during a crisis.

The 2019 coronavirus (COVID-19) pandemic has made the world seem unpredictable. During such crises we can experience concerns that others might be against us, culminating perhaps in paranoid conspiracy theories. Here, we investigate paranoia and belief updating in an online sample (N=1,010) in the United States of America (U.S.A). We demonstrate the pandemic increased individuals' self-rated paranoia and rendered their task-based belief updating more erratic. Local lockdown and reopening policies, as well as culture more broadly, markedly influenced participants' belief-updating: an early and sustained lockdown rendered people's belief updating less capricious. Masks are clearly an effective public health measure against COVID-19. However, state-mandated mask wearing increased paranoia and induced more erratic behaviour. Remarkably, this was most evident in those states where adherence to mask wearing rules was poor but where rule following is typically more common. This paranoia may explain the lack of compliance with this simple and effective countermeasure. Computational analyses of participant behaviour suggested that people with higher paranoia expected the task to be more unstable, but at the same time predicted more rewards. In a follow-up study we found people who were more paranoid endorsed conspiracies about mask-wearing and potential vaccines - again, mask attitude and conspiratorial beliefs were associated with erratic task behaviour and changed priors. Future public health responses to the pandemic might leverage these observations, mollifying paranoia and increasing adherence by tempering people's expectations of other's behaviour, and the environment more broadly, and reinforcing compliance.

Introducing the PLOS ONE Collection on the neuroscience of reward and decision making.

The survival of an organism depends on the ability to make adaptive decisions to achieve the needs of the organism: where to get food, who to mate with, and how to evade predators. Decision-making is a term used to describe a collection of behavioral and/or computational functions that guide the selection of an option amongst a set of alternatives. Some of these functions may include calculating the costs and benefits of a particular action, evaluating differences in value of each of the alternative outcomes and the likelihood of receiving a particular outcome, using past experiences to generate predictions or expectations about action-outcome associations, and/or integration of past experiences to make novel inferences that can be used in new environments. There is considerable interest in understanding the neurobiological mechanisms that mediate these decision-making functions and recent advances in behavioral approaches, neuroscience techniques, and neuroimaging measures have begun to develop mechanistic links between biology, reward, and decision making. This multidisciplinary work holds great promise for elucidating the biological mechanisms mediating decision-making deficits in normal and abnormal states. The multidisciplinary studies included in this Collection provide new insights into the neuroscience of decision making and reward.

Dysregulation of Decision Making Related to Metabotropic Glutamate 5, but Not Midbrain D3, Receptor Availability Following Cocaine Self-administration in Rats.

BACKGROUND: Compulsive patterns of drug use are thought to be the consequence of drug-induced adaptations in the neural mechanisms that enable behavior to be flexible. Neuroimaging studies have found evidence of robust alterations in glutamate and dopamine receptors within brain regions that are known to be critical for decision-making processes in cocaine-dependent individuals, and these changes have been argued to be the consequence of persistent drug use. The causal relationships among drug-induced alterations, cocaine taking, and maladaptive decision-making processes, however, are difficult to establish in humans. METHODS: We assessed decision making in adult male rats using a probabilistic reversal learning task and used positron emission tomography with the [11C]-(+)-PHNO and [18F]FPEB radioligands to quantify regional dopamine D2/3 and metabotropic glutamate 5 (mGlu5) receptor availability, respectively, before and after 21 days of cocaine or saline self-administration. Tests of motivation and relapse-like behaviors were also conducted. RESULTS: We found that self-administration of cocaine, but not of saline, disrupted behavior in the probabilistic reversal learning task measured by selective impairments in negative-outcome updating and also increased cortical mGlu5 receptor availability following 2 weeks of forced abstinence. D2/3 and, importantly, midbrain D3 receptor availability was not altered following 2 weeks of abstinence from cocaine. Notably, the degree of the cocaine-induced increase in cortical mGlu5 receptor availability was related to the degree of disruption in negative-outcome updating. CONCLUSIONS: These findings suggest that cocaine-induced changes in mGlu5 signaling may be a mechanism by which disruptions in negative-outcome updating emerge in cocaine-dependent individuals.

Reinforcement Learning during Adolescence in Rats.

The most dynamic period of postnatal brain development occurs during adolescence, the period between childhood and adulthood. Neuroimaging studies have observed morphologic and functional changes during adolescence, and it is believed that these changes serve to improve the functions of circuits that underlie decision-making. Direct evidence in support of this hypothesis, however, has been limited because most preclinical decision-making paradigms are not readily translated to humans. Here, we developed a reversal-learning protocol for the rapid assessment of adaptive choice behavior in dynamic environments in rats as young as postnatal day 30. A computational framework was used to elucidate the reinforcement-learning mechanisms that change in adolescence and into adulthood. Using a cross-sectional and longitudinal design, we provide the first evidence that value-based choice behavior in a reversal-learning task improves during adolescence in male and female Long-Evans rats and demonstrate that the increase in reversal performance is due to alterations in value updating for positive outcomes. Furthermore, we report that reversal-learning trajectories in adolescence reliably predicted reversal performance in adulthood. This novel behavioral protocol provides a unique platform for conducting biological and systems-level analyses of the neurodevelopmental mechanisms of decision-making.SIGNIFICANCE STATEMENT The neurodevelopmental adaptations that occur during adolescence are hypothesized to underlie age-related improvements in decision-making, but evidence to support this hypothesis has been limited. Here, we describe a novel behavioral protocol for rapidly assessing adaptive choice behavior in adolescent rats with a reversal-learning paradigm. Using a computational approach, we demonstrate that age-related changes in reversal-learning performance in male and female Long-Evans rats are linked to specific reinforcement-learning mechanisms and are predictive of reversal-learning performance in adulthood. Our behavioral protocol provides a unique platform for elucidating key components of adolescent brain function.

Paranoia as a deficit in non-social belief updating.

Paranoia is the belief that harm is intended by others. It may arise from selective pressures to infer and avoid social threats, particularly in ambiguous or changing circumstances. We propose that uncertainty may be sufficient to elicit learning differences in paranoid individuals, without social threat. We used reversal learning behavior and computational modeling to estimate belief updating across individuals with and without mental illness, online participants, and rats chronically exposed to methamphetamine, an elicitor of paranoia in humans. Paranoia is associated with a stronger prior on volatility, accompanied by elevated sensitivity to perceived changes in the task environment. Methamphetamine exposure in rats recapitulates this impaired uncertainty-driven belief updating and rigid anticipation of a volatile environment. Our work provides evidence of fundamental, domain-general learning differences in paranoid individuals. This paradigm enables further assessment of the interplay between uncertainty and belief-updating across individuals and species.

Everyone has had fleeting concerns that others might be against them at some point in their lives. Sometimes these concerns can escalate into paranoia and become debilitating. Paranoia is a common symptom in serious mental illnesses like schizophrenia. It can cause extreme distress and is linked with an increased risk of violence towards oneself or others. Understanding what happens in the brains of people experiencing paranoia might lead to better ways to treat or manage it. Some experts argue that paranoia is caused by errors in the way people assess social situations. An alternative idea is that paranoia stems from the way the brain forms and updates beliefs about the world. Now, Reed et al. show that both people with paranoia and rats exposed to a paranoia-inducing substance expect the world will change frequently, change their minds often, and have a harder time learning in response to changing circumstances. In the experiments, human volunteers with and without psychiatric disorders played a game where the best choices change. Then, the participants completed a survey to assess their level of paranoia. People with higher levels of paranoia predicted more changes would occur and made less predictable choices. In a second set of experiments, rats were put in a cage with three holes where they sometimes received sugar rewards. Some of the rats received methamphetamine, a drug that causes paranoia in humans. Rats given the drug also expected the location of the sugar reward would change often. The drugged animals had harder time learning and adapting to changing circumstances. The experiments suggest that brain processes found in both rats, which are less social than humans, and humans contribute to paranoia. This suggests paranoia may make it harder to update beliefs. This may help scientists understand what causes paranoia and develop therapies or drugs that can reduce paranoia. This information may also help scientists understand why during societal crises like wars or natural disasters humans are prone to believing conspiracies. This is particularly important now as the world grapples with climate change and a global pandemic. Reed et al. note paranoia may impede the coordination of collaborative solutions to these challenging situations.

Medial Nucleus Accumbens Projections to the Ventral Tegmental Area Control Food Consumption.

Decades of research have shown that the NAc is a critical region influencing addiction, mood, and food consumption through its effects on reinforcement learning, motivation, and hedonic experience. Pharmacological studies have demonstrated that inhibition of the NAc shell induces voracious feeding, leading to the hypothesis that the inhibitory projections that emerge from the NAc normally act to restrict feeding. While much of this work has focused on projections to the lateral hypothalamus, the role of NAc projections to the VTA in the control food intake has been largely unexplored. Using a retrograde viral labeling technique and real-time monitoring of neural activity with fiber photometry, we find that medial NAc shell projections to the VTA (mNAc→VTA) are inhibited during food-seeking and food consumption in male mice. We also demonstrate that this circuit bidirectionally controls feeding: optogenetic activation of NAc projections to the VTA inhibits food-seeking and food intake (in both sexes), while optogenetic inhibition of this circuit potentiates food-seeking behavior. Additionally, we show that activity of the NAc to VTA pathway is necessary for adaptive inhibition of food intake in response to external cues. These data provide new insight into NAc control over feeding in mice, and contribute to an emerging literature elucidating the role of inhibitory midbrain feedback within the mesolimbic circuit.SIGNIFICANCE STATEMENT The medial NAc has long been known to control consummatory behavior, with particular focus on accumbens projections to the lateral hypothalamus. Conversely, NAc projections to the VTA have mainly been studied in the context of drug reward. We show that NAc projections to the VTA bidirectionally control food intake, consistent with a permissive role in feeding. Additionally, we show that this circuit is normally inactivated during consumption and food-seeking. Together, these findings elucidate how mesolimbic circuits control food consumption.

Midbrain D3 Receptor Availability Predicts Escalation in Cocaine Self-administration.

BACKGROUND: Results from neuroimaging studies suggest that disruptions in flexible decision-making functions in substance-dependent individuals are a consequence of drug-induced neural adaptations. In addicted populations, however, the causal relationship between biobehavioral phenotypes of susceptibility and addiction consequence is difficult to dissociate. Indeed, evidence from animals suggests that poor decision making due to preexisting biological factors can independently enhance the risk for developing addiction-like behaviors. Neuroimaging studies in animals provide a unique translational approach for the identification of the neurobiological mechanisms that mediate susceptibility to addiction. METHODS: We used positron emission tomography in rats to quantify regional dopamine D2/3 receptors and metabotropic glutamate receptor 5 (mGluR5) and assessed decision making using a probabilistic reversal learning task. Susceptibility to self-administer cocaine was then quantified for 21 days followed by tests of motivation and relapse-like behaviors. RESULTS: We found that deficits specifically in reward-guided choice behavior on the probabilistic reversal learning task predicted greater escalation of cocaine self-administration behavior and greater motivation for cocaine and, critically, were associated with higher midbrain D3 receptor availability. Additionally, individual differences in midbrain D3 receptor availability independently predicted the rate of escalation in cocaine-taking behaviors. No differences in mGluR5 availability, responses during tests of extinction, or cue-induced reinstatement were observed between the groups. CONCLUSIONS: These findings indicate that our identified D3-mediated decision-making phenotype can be used as a behavioral biomarker for assessment of cocaine use susceptibility in human populations.

The Neurobiology of Impulsive Decision-Making and Reinforcement Learning in Nonhuman Animals.

Impulsive decisions are those that favor immediate over delayed rewards, involve the acceptance of undue risk or uncertainty, or fail to adapt to environmental changes. Pathological levels of impulsive decision-making have been observed in individuals with mental illness, but there may be substantial heterogeneity in the processes that drive impulsive choices. Understanding this behavioral heterogeneity may be critical for understanding associated diverseness in the neural mechanisms that give rise to impulsivity. The application of reinforcement learning algorithms in the deconstruction of impulsive decision-making phenotypes can help bridge the gap between biology and behavior and provide insights into the biobehavioral heterogeneity of impulsive choice. This chapter will review the literature on the neurobiological mechanisms of impulsive decision-making in nonhuman animals; specifically, the role of the amine neuromodulatory systems (dopamine, serotonin, norepinephrine, and acetylcholine) in impulsive decision-making and reinforcement learning processes is discussed. Ultimately, the integration of reinforcement learning algorithms with sophisticated behavioral and neuroscience techniques may be critical for advancing the understanding of the neurochemical basis of impulsive decision-making.