The neurobehavioral correlates of exploration without learning: Trading off value for explicit, prospective, and variable information gains.

Exploration is typically motivated by gaining information, with previous research showing that potential information gains drive a "directed" type of exploration. Yet, this research usually studies exploration in the context of learning paradigms and does not directly manipulate multiple levels of information gain. Here, we present a task that isolates learning from decision-making and controls the magnitude of prospective information gains. As predicted, participants explore more with larger future information gains. Both value gains and information gains, at a trial-by-trial level, engage the ventromedial prefrontal cortex (vmPFC), the ventral striatum (VStr), the amygdala, the dorsal anterior cingulate cortex (dACC), and the anterior insula (aINS). Moreover, individual sensitivities to value gains and information gains modulate the vmPFC, dACC, and aINS, but the amygdala and VStr are modulated only by individual sensitivities to information gains. Overall, we identify the neural circuitry of information-based exploration and its relationship with inter-individual exploration biases.

Interleaved Propofol-Ketamine Maintains DBS Physiology and Hemodynamic Stability: A Double-Blind Randomized Controlled Trial.

BACKGROUND: The gold standard anesthesia for deep brain stimulation (DBS) surgery is the "awake" approach, using local anesthesia alone. Although it offers high-quality microelectrode recordings and therapeutic-window assessment, it potentially causes patients extreme stress and might result in suboptimal surgical outcomes. General anesthesia or deep sedation is an alternative, but may reduce physiological testing reliability and lead localization accuracy. OBJECTIVES: The aim is to investigate a novel anesthesia regimen of ketamine-induced conscious sedation for the physiological testing phase of DBS surgery. METHODS: Parkinson's patients undergoing subthalamic DBS surgery were randomly divided into experimental and control groups. During physiological testing, the groups received 0.25 mg/kg/h ketamine infusion and normal saline, respectively. Both groups had moderate propofol sedation before and after physiological testing. The primary outcome was recording quality. Secondary outcomes included hemodynamic stability, lead accuracy, motor and cognitive outcome, patient satisfaction, and adverse events. RESULTS: Thirty patients, 15 from each group, were included. Intraoperatively, the electrophysiological signature and lead localization were similar under ketamine and saline. Tremor amplitude was slightly lower under ketamine. Postoperatively, patients in the ketamine group reported significantly higher satisfaction with anesthesia. The improvement in Unified Parkinson's disease rating scale part-III was similar between the groups. No negative effects of ketamine on hemodynamic stability or cognition were reported perioperatively. CONCLUSIONS: Ketamine-induced conscious sedation provided high quality microelectrode recordings comparable with awake conditions. Additionally, it seems to allow superior patient satisfaction and hemodynamic stability, while maintaining similar post-operative outcomes. Therefore, it holds promise as a novel alternative anesthetic regimen for DBS. © 2024 The Authors. Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.

Irrelevant Threats Linger and Affect Behavior in High Anxiety.

Threat-related information attracts attention and disrupts ongoing behavior, and particularly so for more anxious individuals. Yet, it is unknown how and to what extent threat-related information leave lingering influences on behavior (e.g., by impeding ongoing learning processes). Here, human male and female participants (N = 47) performed probabilistic reinforcement learning tasks where irrelevant distracting faces (neutral, happy, or fearful) were presented together with relevant monetary feedback. Behavioral modeling was combined with fMRI data (N = 27) to explore the neurocomputational bases of learning relevant and irrelevant information. In two separate studies, individuals with high trait anxiety showed increased avoidance of objects previously paired with the combination of neutral monetary feedback and fearful faces (but not neutral or happy faces). Behavioral modeling revealed that high anxiety increased the integration of fearful faces during feedback learning, and fMRI results (regarded as provisional, because of a relatively small sample size) further showed that variance in the prediction error signal, uniquely accounted for by fearful faces, correlated more strongly with activity in the right DLPFC for more anxious individuals. Behavioral and neuronal dissociations indicated that the threat-related distractors did not simply disrupt learning processes. By showing that irrelevant threats exert long-lasting influences on behavior, our results extend previous research that separately showed that anxiety increases learning from aversive feedbacks and distractibility by threat-related information. Our behavioral results, combined with the proposed neurocomputational mechanism, may help explain how increased exposure to irrelevant affective information contributes to the acquisition of maladaptive behaviors in more anxious individuals.SIGNIFICANCE STATEMENT In modern-day society, people are increasingly exposed to various types of irrelevant information (e.g., intruding social media announcements). Yet, the neurocomputational mechanisms influenced by irrelevant information during learning, and their interactions with increasingly distracted personality types are largely unknown. Using a reinforcement learning task, where relevant feedback is presented together with irrelevant distractors (emotional faces), we reveal an interaction between irrelevant threat-related information (fearful faces) and interindividual anxiety levels. fMRI shows provisional evidence for an interaction between anxiety levels and the coupling between activity in the DLPFC and learning signals specifically elicited by fearful faces. Our study reveals how irrelevant threat-related information may become entrenched in the anxious psyche and contribute to long-lasting abnormal behaviors.

Average reward rates enable motivational transfer across independent reinforcement learning tasks.

Outcomes and feedbacks on performance may influence behavior beyond the context in which it was received, yet it remains unclear what neurobehavioral mechanisms may account for such lingering influences on behavior. The average reward rate (ARR) has been suggested to regulate motivated behavior, and was found to interact with dopamine-sensitive cognitive processes, such as vigilance and associative memory encoding. The ARR could therefore provide a bridge between independent tasks when these are performed in temporal proximity, such that the reward rate obtained in one task could influence performance in a second subsequent task. Reinforcement learning depends on the coding of prediction error signals by dopamine neurons and their downstream targets, in particular the nucleus accumbens. Because these brain regions also respond to changes in ARR, reinforcement learning may be vulnerable to changes in ARR. To test this hypothesis, we designed a novel paradigm in which participants (n = 245) performed two probabilistic reinforcement learning tasks presented in interleaved trials. The ARR was controlled by an "induction" task which provided feedback with a low (p = 0.58), a medium (p = 0.75), or a high probability of reward (p = 0.92), while the impact of ARR on reinforcement learning was tested by a second "reference" task with a constant reward probability (p = 0.75). We find that performance was significantly lower in the reference task when the induction task provided low reward probabilities (i.e., during low levels of ARR), as compared to the medium and high ARR conditions. Behavioral modeling further revealed that the influence of ARR is best described by models which accumulates average rewards (rather than average prediction errors), and where the ARR directly modulates the prediction error signal (rather than affecting learning rates or exploration). Our results demonstrate how affective information in one domain may transfer and affect motivated behavior in other domains. These findings are particularly relevant for understanding mood disorders, but may also inform abnormal behaviors attributed to dopamine dysfunction.

The Neural Representations of Emotional Experiences Are More Similar Than Those of Neutral Experiences.

Stimuli that evoke the same feelings can nevertheless look different and have different semantic meanings. Although we know much about the neural representation of emotion, the neural underpinnings of emotional similarity are unknown. One possibility is that the same brain regions represent similarity between emotional and neutral stimuli, perhaps with different strengths. Alternatively, emotional similarity could be coded in separate regions, possibly those sensitive to emotional valence and arousal. In behavior, the extent to which people consider similarity along emotional dimensions when they evaluate the overall similarity between stimuli has never been investigated. Although the emotional features of stimuli may dominate explicit ratings of similarity, it is also possible that people neglect emotional dimensions as irrelevant to that judgment. We contrasted these hypotheses in (male and female) healthy controls using two measures of similarity and two picture databases of complex negative and neutral scenes, the second of which provided exquisite control over semantic and visual attributes. The similarity between emotional stimuli was greater than between neutral stimuli in the inferior temporal cortex, the fusiform face area, and the precuneus. Additionally, only the similarity between emotional stimuli was significantly represented in early visual cortex, anterior insula and dorsal anterior cingulate cortex. Intriguingly, despite the stronger neural similarity between emotional stimuli, the same participants did not rate them as more similar to each other than neutral stimuli. These results contribute to our understanding of how emotion is represented within a general conceptual workspace and of the overgeneralization bias in anxiety disorders.SIGNIFICANCE STATEMENT We tested differences in similarity between emotional and neutral scenes. Arousal and negative valence did not increase similarity ratings. When conditions were equated on semantic similarity, participants rated emotional stimuli as similar to each other as neutral ones. Despite this equivalence, the similarity among the neural representations of emotional compared with neutral stimuli was higher in regions, which also expressed similarity between neutral stimuli and in unique regions. We report a striking difference between behavioral and neural similarity; strong neural similarity between emotional pictures did not influence similarity judgements in the same participants in the behavioral rating task after the scan. These findings may have an impact on research about the neural representations of emotional categories and the overgeneralization bias in anxiety disorders.

A neural and behavioral trade-off between value and uncertainty underlies exploratory decisions in normative anxiety.

Exploration reduces uncertainty about the environment and improves the quality of future decisions, but at the cost of provisional uncertain and suboptimal outcomes. Although anxiety promotes intolerance to uncertainty, it remains unclear whether and by which mechanisms anxiety relates to exploratory decision-making. We use a dynamic three-armed-bandit task and find that higher trait-anxiety is associated with increased exploration, which in turn harms overall performance. We identify two distinct behavioral sources: first, decisions made by anxious individuals are guided toward reduction of uncertainty; and second, decisions are less guided by immediate value gains. These findings are similar in both loss and gain domains, and further demonstrate that an affective trait relates to exploration and results in an inverse-U-shaped relationship between anxiety and overall performance. Additional imaging data (fMRI) suggests that normative anxiety correlates negatively with the representation of expected-value in the dorsal-anterior-cingulate-cortex, and in contrast, positively with the representation of uncertainty in the anterior-insula. We conclude that a trade-off between value-gains and uncertainty-reduction entails maladaptive decision-making in individuals with higher normal-range anxiety.

Quantifying the excitatory-inhibitory balance: A comparison of SemiLASER and MEGA-SemiLASER for simultaneously measuring GABA and glutamate at 7T.

The importance of the excitatory-inhibitory (E/I) balance in a wide range of cognitive and behavioral processes has prompted a commensurate interest in methods for reliably quantifying it. Proton Magnetic Resonance Spectroscopy (1H-MRS) remains the only method capable of safely and non-invasively measuring the concentrations of the brain's major excitatory (glutamate) and inhibitory (γ-aminobutyric-acid, GABA) neurotransmitters in-vivo. MRS relies on spectral Mescher-Garwood (MEGA) editing techniques at 3T to distinguish GABA from its overlapping resonances. However, with the increased spectral resolution at ultrahigh field strengths of 7T and above, non-edited spectroscopic techniques become potential viable alternatives to MEGA based approaches, and also address some of their shortcomings, such as signal loss, sensitivity to transmitter inhomogeneities and temporal resolution. We present a comprehensive comparison of both edited and non-edited strategies at 7T for simultaneously quantifying glutamate and GABA from the dorsal anterior cingulate cortex (dACC), and evaluate their reproducibility and relative bias. The combined root-mean-square test-retest reproducibility of Glu and GABA (CVE/I) was as low as 13.3% for unedited MRS at TE=80 ms using SemiLASER localization, while edited MRS at TE=80 ms yielded CVE/I=20% and 21% for asymmetric and symmetric MEGA editing, respectively. An unedited SemiLASER acquisition using a shorter echo time of TE=42 ms yielded CVE/I as low as 24.9%. Our results show that non-edited sequences at an echo time of 80 ms provide better reproducibility than either edited sequences at the same TE, or non-edited sequences at a shorter TE of 42 ms. This is supported by numerical simulations and is driven in part by a pseudo-singlet appearance of the GABA multiplets at TE=80 ms, and the excellent spectral resolution at 7T. Our results uphold a transition to non-edited MRS for monitoring the E/I balance at ultrahigh fields, and stress the importance of using a properly-optimized echo time.

Two sides of the same amygdala: From shared neural mechanisms to comorbidity of social and affective disorders.

Recent studies identified a circuitry within the primate amygdala that underlies both social and affective processes. Such shared functions within the same circuit, although beneficial for adaptive behavior and make sense in light of evolution, can also contribute to the growing comorbidity between affective and social disorders.

Automatic Recognition of Macaque Facial Expressions for Detection of Affective States.

Internal affective states produce external manifestations such as facial expressions. In humans, the Facial Action Coding System (FACS) is widely used to objectively quantify the elemental facial action units (AUs) that build complex facial expressions. A similar system has been developed for macaque monkeys-the Macaque FACS (MaqFACS); yet, unlike the human counterpart, which is already partially replaced by automatic algorithms, this system still requires labor-intensive coding. Here, we developed and implemented the first prototype for automatic MaqFACS coding. We applied the approach to the analysis of behavioral and neural data recorded from freely interacting macaque monkeys. The method achieved high performance in the recognition of six dominant AUs, generalizing between conspecific individuals (Macaca mulatta) and even between species (Macaca fascicularis). The study lays the foundation for fully automated detection of facial expressions in animals, which is crucial for investigating the neural substrates of social and affective states.

Neurons in the Nonhuman Primate Amygdala and Dorsal Anterior Cingulate Cortex Signal Aversive Memory Formation under Sedation.

BACKGROUND: Anesthetics aim to prevent memory of unpleasant experiences. The amygdala and dorsal anterior cingulate cortex participate in forging emotional and valence-driven memory formation. It was hypothesized that this circuitry maintains its role under sedation. METHODS: Two nonhuman primates underwent aversive tone-odor conditioning under sedative states induced by ketamine or midazolam (1 to 8 and 0.1 to 0.8 mg/kg, respectively). The primary outcome was behavioral and neural evidence suggesting memory formation. This study simultaneously measured conditioned inspiratory changes and changes in firing rate of single neurons in the amygdala and the dorsal anterior cingulate cortex in response to an expected aversive olfactory stimulus appearing during acquisition and tested their retention after recovery. RESULTS: Aversive memory formation occurred in 26 of 59 sessions under anesthetics (16 of 29 and 10 of 30, 5 of 30 and 21 of 29 for midazolam and ketamine at low and high doses, respectively). Single-neuron responses in the amygdala and dorsal anterior cingulate cortex were positively correlated between acquisition and retention (amygdala, n = 101, r = 0.51, P < 0.001; dorsal anterior cingulate cortex, n = 121, r = 0.32, P < 0.001). Neural responses during acquisition under anesthetics were stronger in sessions exhibiting memory formation than those that did not (amygdala median response ratio, 0.52 versus 0.33, n = 101, P = 0.021; dorsal anterior cingulate cortex median response ratio, 0.48 versus 0.32, n = 121, P = 0.012). The change in firing rate of amygdala neurons during acquisition was correlated with the size of stimuli-conditioned inspiratory response during retention (n = 101, r = 0.22 P = 0.026). Thus, amygdala and dorsal anterior cingulate cortex responses during acquisition under anesthetics predicted retention. Respiratory unconditioned responses to the aversive odor anesthetics did not differ from saline controls. CONCLUSIONS: These results suggest that the amygdala-dorsal anterior cingulate cortex circuit maintains its role in acquisition and maintenance of aversive memories in nonhuman primates under sedation with ketamine and midazolam and that the stimulus valence is sufficient to drive memory formation.

The geometry of neuronal representations during rule learning reveals complementary roles of cingulate cortex and putamen.

Learning new rules and adopting novel behavioral policies is a prominent adaptive behavior of primates. We studied the dynamics of single neurons in the dorsal anterior cingulate cortex and putamen of monkeys while they learned new classification tasks every few days over a fixed set of multi-cue patterns. Representing the rules and the neuronal selectivity as vectors in the space spanned by a set of stimulus features allowed us to characterize neuronal dynamics in geometrical terms. We found that neurons in the cingulate cortex mainly rotated toward the rule, implying a policy search, whereas neurons in the putamen showed a magnitude increase that followed the rotation of cortical neurons, implying strengthening of confidence for the newly acquired rule-based policy. Further, the neural representation at the end of a session predicted next-day behavior, reflecting overnight retention. The novel framework for characterization of neural dynamics suggests complementing roles for the putamen and the anterior cingulate cortex.

Shared yet dissociable neural codes across eye gaze, valence and expectation.

The direction of the eye gaze of others is a prominent social cue in primates and is important for communication1-11. Although gaze can signal threat and elicit anxiety6,12,13, it remains unclear whether it shares neural circuitry with stimulus value. Notably, gaze not only has valence, but can also serve as a predictor of the outcome of a social encounter, which can be either negative or positive2,8,12,13. Here we show that the neural codes for gaze and valence overlap in primates and that they involve two different mechanisms: one for the outcome and another for its expectation. Monkeys participated in the human intruder test13,14, in which a human participant had either a direct or averted gaze, interleaved with blocks of aversive and appetitive conditioning. We find that single neurons in the amygdala encode gaze15, whereas neurons in the anterior cingulate cortex encode the social context16, but not gaze. We identify a shared population in the amygdala for which the neural responses to direct and averted gaze parallel the responses to aversive and appetitive stimulus, respectively. Furthermore, we distinguish between two neural mechanisms-an overall-activity scheme that is used for gaze and the unconditioned stimulus, and a correlated-selectivity scheme that is used for gaze and the conditioned stimulus. These findings provide insights into the origins of the neural mechanisms that underlie the computations of both social interactions and valence, and could help to shed light on mechanisms that underlie social anxiety and the comorbidity between anxiety and impaired social interactions.

Prediction errors bidirectionally bias time perception.

Time perception and prediction errors are essential for everyday life. We hypothesized that their putative shared circuitry in the striatum might enable these two functions to interact. We show that positive and negative prediction errors bias time perception by increasing and decreasing perceived time, respectively. Imaging and behavioral modeling identify this interaction to occur in the putamen. Depending on context, this interaction may have beneficial or adverse effects.

Local Targeted Memory Reactivation in Human Sleep.

Memory consolidation can be promoted via targeted memory reactivation (TMR) that re-presents training cues or context during sleep. Whether TMR acts locally or globally on cortical sleep oscillations remains unknown. Here, we exploit the unique functional neuroanatomy of olfaction with its ipsilateral stimulus processing to perform local TMR in one brain hemisphere. Participants learned associations between words and locations in left or right visual fields with contextual odor throughout. We found lateralized event-related potentials during task training that indicate unihemispheric memory processes. During post-learning naps, odors were presented to one nostril in non-rapid eye movement (NREM) sleep. Memory for specific words processed in the cued hemisphere (ipsilateral to stimulated nostril) was improved after local TMR during sleep. Unilateral odor cues locally modulated slow-wave (SW) power such that regional SW power increase was lower in the cued hemisphere relative to the uncued hemisphere and negatively correlated with select memories for cued words. Moreover, local TMR improved phase-amplitude coupling (PAC) between slow oscillations and sleep spindles specifically in the cued hemisphere. The effects on memory performance and cortical sleep oscillations were not observed when unilateral olfactory stimulation during sleep followed learning without contextual odor. Thus, TMR in human sleep transcends global action by selectively promoting specific memories associated with local sleep oscillations.

Prestimulus Activity in the Cingulo-Opercular Network Predicts Memory for Naturalistic Episodic Experience.

Human memory is strongly influenced by brain states occurring before an event, yet we know little about the underlying mechanisms. We found that activity in the cingulo-opercular network (including bilateral anterior insula [aI] and anterior prefrontal cortex [aPFC]) seconds before an event begins can predict whether this event will subsequently be remembered. We then tested how activity in the cingulo-opercular network shapes memory performance. Our findings indicate that prestimulus cingulo-opercular activity affects memory performance by opposingly modulating subsequent activity in two sets of regions previously linked to encoding and retrieval of episodic information. Specifically, higher prestimulus cingulo-opercular activity was associated with a subsequent increase in activity in temporal regions previously linked to encoding and with a subsequent reduction in activity within a set of regions thought to play a role in retrieval and self-referential processing. Together, these findings suggest that prestimulus attentional states modulate memory for real-life events by enhancing encoding and possibly by dampening interference from competing memory substrates.

Affective memory rehearsal with temporal sequences in amygdala neurons.

Affective learning and memory are essential for daily behavior, with both adaptive and maladaptive learning depending on stimulus-evoked activity in the amygdala circuitry. Behavioral studies further suggest that post-association offline processing contributes to memory formation. Here we investigated spike sequences across simultaneously recorded neurons while monkeys learned to discriminate between aversive and pleasant tone-odor associations. We show that triplets of neurons exhibit consistent temporal sequences of spiking activity that differed from firing patterns of individual neurons and pairwise correlations. These sequences occurred throughout the long post-trial period, contained valence-related information, declined as learning progressed and were selectively present in activity evoked by the recent pairing of a conditioned stimulus with an unconditioned stimulus. Our findings reveal that temporal sequences across neurons in the primate amygdala serve as a coding mechanism and might aid memory formation through the rehearsal of the recently experienced association.

Re-exploring Mechanisms of Exploration.

Deciding when to exploit what is already known and when to explore new possibilities is crucial for adapting to novel and dynamic environments. Using reinforcement-based decision making, Costa et al. (2019) in this issue of Neuron find that neurons in the amygdala and ventral-striatum differentially signal the benefit from exploring new options and exploiting familiar ones.

A Tradeoff in the Neural Code across Regions and Species.

Many evolutionary years separate humans and macaques, and although the amygdala and cingulate cortex evolved to enable emotion and cognition in both, an evident functional gap exists. Although they were traditionally attributed to differential neuroanatomy, functional differences might also arise from coding mechanisms. Here we find that human neurons better utilize information capacity (efficient coding) than macaque neurons in both regions, and that cingulate neurons are more efficient than amygdala neurons in both species. In contrast, we find more overlap in the neural vocabulary and more synchronized activity (robustness coding) in monkeys in both regions and in the amygdala of both species. Our findings demonstrate a tradeoff between robustness and efficiency across species and regions. We suggest that this tradeoff can contribute to differential cognitive functions between species and underlie the complementary roles of the amygdala and the cingulate cortex. In turn, it can contribute to fragility underlying human psychopathologies.