Contribution of amygdala to dynamic model arbitration under uncertainty.

Intrinsic uncertainty in the reward environment requires the brain to run multiple models simultaneously to predict outcomes based on preceding cues or actions, commonly referred to as stimulus- and action-based learning. Ultimately, the brain also must adopt appropriate choice behavior using reliability of these models. Here, we combined multiple experimental and computational approaches to quantify concurrent learning in monkeys performing tasks with different levels of uncertainty about the model of the environment. By comparing behavior in control monkeys and monkeys with bilateral lesions to the amygdala or ventral striatum, we found evidence for dynamic, competitive interaction between stimulus-based and action-based learning, and for a distinct role of the amygdala. Specifically, we demonstrate that the amygdala adjusts the initial balance between the two learning systems, thereby altering the interaction between arbitration and learning that shapes the time course of both learning and choice behaviors. This novel role of the amygdala can account for existing contradictory observations and provides testable predictions for future studies into circuit-level mechanisms of flexible learning and choice under uncertainty.

Lesions to the mediodorsal thalamus, but not orbitofrontal cortex, enhance volatility beliefs linked to paranoia.

Beliefs-attitudes toward some state of the environment-guide action selection and should be robust to variability but sensitive to meaningful change. Beliefs about volatility (expectation of change) are associated with paranoia in humans, but the brain regions responsible for volatility beliefs remain unknown. The orbitofrontal cortex (OFC) is central to adaptive behavior, whereas the magnocellular mediodorsal thalamus (MDmc) is essential for arbitrating between perceptions and action policies. We assessed belief updating in a three-choice probabilistic reversal learning task following excitotoxic lesions of the MDmc (n = 3) or OFC (n = 3) and compared performance with that of unoperated monkeys (n = 14). Computational analyses indicated a double dissociation: MDmc, but not OFC, lesions were associated with erratic switching behavior and heightened volatility belief (as in paranoia in humans), whereas OFC, but not MDmc, lesions were associated with increased lose-stay behavior and reward learning rates. Given the consilience across species and models, these results have implications for understanding paranoia.

Transcriptomic landscape of mammalian ventral pallidum at single-cell resolution.

The ventral pallidum (VP) is critical for motivated behaviors. While contemporary work has begun to elucidate the functional diversity of VP neurons, the molecular heterogeneity underlying this functional diversity remains incompletely understood. We used snRNA-seq and in situ hybridization to define the transcriptional taxonomy of VP cell types in mice, macaques, and baboons. We found transcriptional conservation between all three species, within the broader neurochemical cell types. Unique dopaminoceptive and cholinergic subclusters were identified and conserved across both primate species but had no homolog in mice. This harmonized consensus VP cellular atlas will pave the way for understanding the structure and function of the VP and identified key neuropeptides, neurotransmitters, and neuro receptors that could be targeted within specific VP cell types for functional investigations.

Motor System-Dependent Effects of Amygdala and Ventral Striatum Lesions on Explore-Exploit Behaviors.

Deciding whether to forego immediate rewards or explore new opportunities is a key component of flexible behavior and is critical for the survival of the species. Although previous studies have shown that different cortical and subcortical areas, including the amygdala and ventral striatum (VS), are implicated in representing the immediate (exploitative) and future (explorative) value of choices, the effect of the motor system used to make choices has not been examined. Here, we tested male rhesus macaques with amygdala or VS lesions on two versions of a three-arm bandit task where choices were registered with either a saccade or an arm movement. In both tasks we presented the monkeys with explore-exploit tradeoffs by periodically replacing familiar options with novel options that had unknown reward probabilities. We found that monkeys explored more with saccades but showed better learning with arm movements. VS lesions caused the monkeys to be more explorative with arm movements and less explorative with saccades, although this may have been due to an overall decrease in performance. VS lesions affected the monkeys' ability to learn novel stimulus-reward associations in both tasks, while after amygdala lesions this effect was stronger when choices were made with saccades. Further, on average, VS and amygdala lesions reduced the monkeys' ability to choose better options only when choices were made with a saccade. These results show that learning reward value associations to manage explore-exploit behaviors is motor system dependent and they further define the contributions of amygdala and VS to reinforcement learning.

The amygdala is not necessary for the familiarity aspect of recognition memory.

Dual-process accounts of item recognition posit two memory processes: slow but detailed recollection, and quick but vague familiarity. It has been proposed, based on prior rodent work, that the amygdala is critical for the familiarity aspect of item recognition. Here, we evaluated this proposal in male rhesus monkeys (Macaca mulatta) with selective bilateral excitotoxic amygdala damage. We used four established visual memory tests designed to assess different aspects of familiarity, all administered on touchscreen computers. Specifically, we assessed monkeys' tendencies to make low-latency false alarms, to make false alarms to recently seen lures, to produce curvilinear ROC curves, and to discriminate stimuli based on repetition across days. Three of the four tests showed no familiarity impairment and the fourth was explained by a deficit in reward processing. Consistent with this, amygdala damage did produce an anticipated deficit in reward processing in a three-arm-bandit gambling task, verifying the effectiveness of the lesions. Together, these results contradict prior rodent work and suggest that the amygdala is not critical for the familiarity aspect of item recognition.

Efficient viral expression of a chemogenetic receptor in the old-world monkey amygdala.

Genetically encoded synthetic receptors, such as the chemogenetic and optogenetic proteins, are powerful tools for functional brain studies in animals. In the primate brain, with its comparatively large, intricate anatomical structures, it can be challenging to express transgenes, such as the hM4Di chemogenetic receptor, in a defined anatomical structure with high penetrance. Here, we compare parameters for lentivirus vector injections in the rhesus monkey amygdala. We find that four injections of 20 µl, infused at 0.5 µl/min, can achieve neuronal hM4Di expression in 50-100% of neurons within a 60 mm3 volume, without observable damage from overexpression. Increasing the number of hM4Di_CFP lentivirus injections to up to 12 sites per hemisphere, resulted in 30%-40% neuronal coverage of the overall amygdala volume, with coverage reaching 60% in some subnuclei. Manganese Chloride was mixed with lentivirus and used as an MRI marker to verify targeting accuracy and correct unsuccessful injections in these experiments. In a separate monkey we visualized, in vivo, viral expression of the hM4Di receptor protein in the amygdala, using Positron Emission Tomography. Together, these data show efficient and verifiable expression of a chemogenetic receptor in old-world monkey amygdala.

A Computational Model Reveals Learning Dynamics During Interpretation Bias Training With Clinical Applications.

BACKGROUND: Some psychopathologies, including anxiety and irritability, are associated with biases when judging ambiguous social stimuli. Interventions targeting these biases, or interpretation bias training (IBT), are amenable to computational modeling to describe their associative learning mechanisms. Here, we translated ALCOVE (attention learning covering map), a model of category learning, to describe learning in youths with affective psychopathology when training on more positive judgments of ambiguous face emotions. METHODS: A predominantly clinical sample comprised 71 youths (age range, 8-22 years) representing broad distributions of irritability and anxiety symptoms. Of these, 63 youths were included in the test sample by completing an IBT task with acceptable performance for computational modeling. We used a separate sample of 28 youths to translate ALCOVE for individual estimates of learning rate and generalization. In the test sample, we assessed associations between model learning estimates and irritability, anxiety, their shared variance (negative affectivity), and age. RESULTS: Age and affective symptoms were associated with category learning during IBT. Lower learning rates were associated with higher negative affectivity common in anxiety and irritability. Lower generalization, or improved discrimination between face emotions, was associated with increasing age. CONCLUSIONS: This work demonstrates a functional consequence of age- and symptom-related learning during interpretation bias. Learning measured by ALCOVE also revealed learning types not accounted for in the prior literature on IBT. This work more broadly demonstrates the utility of measurement models for understanding trial-by-trial processes and identifying individual learning styles.

Adverse childhood experiences and hormonal contraception: Interactive impact on sexual reward function.

The current literature suggests that some women are uniquely vulnerable to negative effects of hormonal contraception (HC) on affective processes. However, little data exists as to which factors contribute to such vulnerability. The present study evaluated the impact of prepubertal adverse childhood experiences (ACEs) on reward processing in women taking HC (N = 541) compared to naturally cycling women (N = 488). Participants completed an online survey assessing current and past HC use and exposure to 10 different adverse childhood experiences (ACEs) before puberty (ACE Questionnaire), with participants categorized into groups of low (0-1) versus high (≥2) prepubertal ACE exposure. Participants then completed a reward task rating their expected and experienced valence for images that were either erotic, pleasant (non-erotic), or neutral. Significant interactions emerged between prepubertal ACE exposure and HC use on expected (p = 0.028) and experienced (p = 0.025) valence ratings of erotic images but not pleasant or neutral images. Importantly, follow-up analyses considering whether women experienced HC-induced decreases in sexual desire informed the significant interaction for expected valence ratings of erotic images. For current HC users, prepubertal ACEs interacted with HC-induced decreased sexual desire (p = 0.008), such that high ACE women reporting decreased sexual desire on HC showed substantially decreased ratings for anticipated erotic images compared to both high prepubertal ACE women without decreased sexual desire (p < 0.001) and low prepubertal ACE women also reporting decreased sexual desire (p = 0.010). The interaction was not significant in naturally cycling women reporting previous HC use, suggesting that current HC use could be impacting anticipatory reward processing of sexual stimuli among certain women (e.g., high prepubertal ACE women reporting HC-induced decreases in sexual desire). The study provides rationale for future randomized, controlled trials to account for prepubertal ACE exposure to promote contraceptive selection informed by behavioral evidence.

Recurrent activity within microcircuits of macaque dorsolateral prefrontal cortex tracks cognitive flexibility.

Human and non-human primate data clearly implicate the dorsolateral prefrontal cortex (dlPFC) as critical for advanced cognitive functions 1,2 . It is thought that intracortical synaptic architectures within dlPFC are the integral neurobiological substrate that gives rise to these processes, including working memory, inferential reasoning, and decision-making 3-7 . In the prevailing model, each cortical column makes up one fundamental processing unit composed of dense intrinsic connectivity, conceptualized as the 'canonical' cortical microcircuit 3,8 . Each cortical microcircuit receives sensory and cognitive information from a variety of sources which are represented by sustained activity within the microcircuit, referred to as persistent or recurrent activity 4,9 . Via recurrent connections within the microcircuit, activity can propagate for a variable length of time, thereby allowing temporary storage and computations to occur locally before ultimately passing a transformed representation to a downstream output 4,5,10 . Competing theories regarding how microcircuit activity is coordinated have proven difficult to reconcile in vivo where intercortical and intracortical computations cannot be fully dissociated 5,9,11,12 . Here, we interrogated the intrinsic features of isolated microcircuit networks using high-density calcium imaging of macaque dlPFC ex vivo . We found that spontaneous activity is intrinsically maintained by microcircuit architecture, persisting at a high rate in the absence of extrinsic connections. Further, using perisulcal stimulation to evoke persistent activity in deep layers, we found that activity propagates through stochastically assembled intracortical networks, creating predictable population-level events from largely non-overlapping ensembles. Microcircuit excitability covaried with individual cognitive performance, thus anchoring heuristic models of abstract cortical functions within quantifiable constraints imposed by the underlying synaptic architecture.

Electrophysiological Markers of Aberrant Cue-Specific Exploration in Hazardous Drinkers.

Background: Hazardous drinking is associated with maladaptive alcohol-related decision-making. Existing studies have often focused on how participants learn to exploit familiar cues based on prior reinforcement, but little is known about the mechanisms that drive hazardous drinkers to explore novel alcohol cues when their value is not known. Methods: We investigated exploration of novel alcohol and non-alcohol cues in hazardous drinkers (N = 27) and control participants (N = 26) during electroencephalography (EEG). A normative computational model with two free parameters was fit to estimate participants' weighting of the future value of exploration and immediate value of exploitation. Results: Hazardous drinkers demonstrated increased exploration of novel alcohol cues, and conversely, increased probability of exploiting familiar alternatives instead of exploring novel non-alcohol cues. The motivation to explore novel alcohol stimuli in hazardous drinkers was driven by an elevated relative future valuation of uncertain alcohol cues. P3a predicted more exploratory decision policies driven by an enhanced relative future valuation of novel alcohol cues. P3b did not predict choice behavior, but computational parameter estimates suggested that hazardous drinkers with enhanced P3b to alcohol cues were likely to learn to exploit their immediate expected value. Conclusions: Hazardous drinkers did not display atypical choice behavior, different P3a/P3b amplitudes, or computational estimates to novel non-alcohol cues-diverging from previous studies in addiction showing atypical generalized explore-exploit decisions with non-drug-related cues. These findings reveal that cue-specific neural computations may drive aberrant alcohol-related decision-making in hazardous drinkers-highlighting the importance of drug-relevant cues in studies of decision-making in addiction.

What Does the Frontopolar Cortex Contribute to Goal-Directed Cognition and Action?

Understanding the unique functions of different subregions of primate prefrontal cortex has been a longstanding goal in cognitive neuroscience. Yet, the anatomy and function of one of its largest subregions (the frontopolar cortex) remain enigmatic and underspecified. Our Society for Neuroscience minisymposium Primate Frontopolar Cortex: From Circuits to Complex Behaviors will comprise a range of new anatomic and functional approaches that have helped to clarify the basic circuit anatomy of the frontal pole, its functional involvement during performance of cognitively demanding behavioral paradigms in monkeys and humans, and its clinical potential as a target for noninvasive brain stimulation in patients with brain disorders. This review consolidates knowledge about the anatomy and connectivity of frontopolar cortex and provides an integrative summary of its function in primates. We aim to answer the question: what, if anything, does frontopolar cortex contribute to goal-directed cognition and action?

The neurocomputational bases of explore-exploit decision-making.

Flexible decision-making requires animals to forego immediate rewards (exploitation) and try novel choice options (exploration) to discover if they are preferable to familiar alternatives. Using the same task and a partially observable Markov decision process (POMDP) model to quantify the value of choices, we first determined that the computational basis for managing explore-exploit tradeoffs is conserved across monkeys and humans. We then used fMRI to identify where in the human brain the immediate value of exploitative choices and relative uncertainty about the value of exploratory choices were encoded. Consistent with prior neurophysiological evidence in monkeys, we observed divergent encoding of reward value and uncertainty in prefrontal and parietal regions, including frontopolar cortex, and parallel encoding of these computations in motivational regions including the amygdala, ventral striatum, and orbitofrontal cortex. These results clarify the interplay between prefrontal and motivational circuits that supports adaptive explore-exploit decisions in humans and nonhuman primates.

Differential coding of goals and actions in ventral and dorsal corticostriatal circuits during goal-directed behavior.

Goal-directed behavior requires identifying objects in the environment that can satisfy internal needs and executing actions to obtain those objects. The current study examines ventral and dorsal corticostriatal circuits that support complementary aspects of goal-directed behavior. We analyze activity from the amygdala, ventral striatum, orbitofrontal cortex, and lateral prefrontal cortex (LPFC) while monkeys perform a three-armed bandit task. Information about chosen stimuli and their value is primarily encoded in the amygdala, ventral striatum, and orbitofrontal cortex, while the spatial information is primarily encoded in the LPFC. Before the options are presented, information about the to-be-chosen stimulus is represented in the amygdala, ventral striatum, and orbitofrontal cortex; at the time of choice, the information is passed to the LPFC to direct a saccade. Thus, learned value information specifying behavioral goals is maintained throughout the ventral corticostriatal circuit, and it is routed through the dorsal circuit at the time actions are selected.

A pragmatic reevaluation of the efficacy of nonhuman primate optogenetics for psychiatry.

Translational neuroscience is committed to generating discoveries in the laboratory that ultimately can improve human lives. Optogenetics has received considerable attention because of its demonstrated promise in rodent brains to manipulate cells and circuits. In a recent report, Tremblay et al. [28] introduce an open resource detailing optogenetic studies of the nonhuman primate (NHP) brain and make robust claims about the translatability of the technology. We propose that their quantitative (e.g. a 91% success rate) and theoretical claims are questionable because the data were analyzed at a level relevant to the rodent but not NHP brain. Injections were clustered within a few monkeys in a few studies in a few brain regions, and their definitions of success were not clearly relevant to human neuropsychiatric disease. A reanalysis of the data with a modified definition of success that included a behavioral and biological effect revealed a 62.5% success rate that was lower when considering only strong outcomes (53.1%). This calls into question the current efficacy of optogenetic techniques in the NHP brain and suggests that we are a long way from being able to leverage them in 'the service of patients with neurological or psychiatric conditions' as the Tremblay report claims.

Entropy-based metrics for predicting choice behavior based on local response to reward.

For decades, behavioral scientists have used the matching law to quantify how animals distribute their choices between multiple options in response to reinforcement they receive. More recently, many reinforcement learning (RL) models have been developed to explain choice by integrating reward feedback over time. Despite reasonable success of RL models in capturing choice on a trial-by-trial basis, these models cannot capture variability in matching behavior. To address this, we developed metrics based on information theory and applied them to choice data from dynamic learning tasks in mice and monkeys. We found that a single entropy-based metric can explain 50% and 41% of variance in matching in mice and monkeys, respectively. We then used limitations of existing RL models in capturing entropy-based metrics to construct more accurate models of choice. Together, our entropy-based metrics provide a model-free tool to predict adaptive choice behavior and reveal underlying neural mechanisms.

Adolescent Dopamine Neurons Represent Reward Differently during Action and State Guided Learning.

Neuronal underpinning of learning cause-and-effect associations in the adolescent brain remains poorly understood. Two fundamental forms of associative learning are Pavlovian (classical) conditioning, where a stimulus is followed by an outcome, and operant (instrumental) conditioning, where outcome is contingent on action execution. Both forms of learning, when associated with a rewarding outcome, rely on midbrain dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SN). We find that, in adolescent male rats, reward-guided associative learning is encoded differently by midbrain dopamine neurons in each conditioning paradigm. Whereas simultaneously recorded VTA and SN adult neurons have a similar phasic response to reward delivery during both forms of conditioning, adolescent neurons display a muted reward response during operant but a profoundly larger reward response during Pavlovian conditioning. These results suggest that adolescent neurons assign a different value to reward when it is not gated by action. The learning rate of adolescents and adults during both forms of conditioning was similar, supporting the notion that differences in reward response in each paradigm may be because of differences in motivation and independent of state versus action value learning. Static characteristics of dopamine neurons, such as dopamine cell number and size, were similar in the VTA and SN of both ages, but there were age-related differences in stimulated dopamine release and correlated spike activity, suggesting that differences in reward responsiveness by adolescent dopamine neurons are not because of differences in intrinsic properties of these neurons but engagement of different dopaminergic networks.SIGNIFICANCE STATEMENT Reckless behavior and impulsive decision-making by adolescents suggest that motivated behavioral states are encoded differently by the adolescent brain. Motivated behavior, which is dependent on the function of the dopamine system, follows learning of cause-and-effect associations in the environment. We find that dopamine neurons in adolescents encode reward differently depending on the cause-and-effect relationship of the means to receive that reward. Compared with adults, reward contingent on action led to a muted response, whereas reward that followed a cue but was not gated by action produced an augmented phasic response. These data demonstrate an age-related difference in dopamine neuron response to reward that is not uniform and is guided by processes that differentiate between state and action values.

Anterior cingulate and putamen neurons flexibly learn whether a hot dog is a sandwich.

Policy search lets you discover rules and adapt behavior. In this issue of Neuron, Cohen et al. (2021) demonstrate that the dynamics of neurons in primate anterior cingulate cortex and putamen indicate when a correct policy is discovered and confidence in executing decisions under that policy.