Heterogeneity in strategy use during arbitration between experiential and observational learning.

To navigate our complex social world, it is crucial to deploy multiple learning strategies, such as learning from directly experiencing action outcomes or from observing other people's behavior. Despite the prevalence of experiential and observational learning in humans and other social animals, it remains unclear how people favor one strategy over the other depending on the environment, and how individuals vary in their strategy use. Here, we describe an arbitration mechanism in which the prediction errors associated with each learning strategy influence their weight over behavior. We designed an online behavioral task to test our computational model, and found that while a substantial proportion of participants relied on the proposed arbitration mechanism, there was some meaningful heterogeneity in how people solved this task. Four other groups were identified: those who used a fixed mixture between the two strategies, those who relied on a single strategy and non-learners with irrelevant strategies. Furthermore, groups were found to differ on key behavioral signatures, and on transdiagnostic symptom dimensions, in particular autism traits and anxiety. Together, these results demonstrate how large heterogeneous datasets and computational methods can be leveraged to better characterize individual differences.

Optogenetic Inhibition of Rat Anterior Cingulate Cortex Impairs the Ability to Initiate and Stay on Task.

Our prior research has identified neural correlates of cognitive control in the anterior cingulate cortex (ACC), leading us to hypothesize that the ACC is necessary for increasing attention as rats flexibly learn new contingencies during a complex reward-guided decision-making task. Here, we tested this hypothesis by using optogenetics to transiently inhibit the ACC, while rats of either sex performed the same two-choice task. ACC inhibition had a profound impact on behavior that extended beyond deficits in attention during learning when expected outcomes were uncertain. We found that ACC inactivation slowed and reduced the number of trials rats initiated and impaired both their accuracy and their ability to complete sessions. Furthermore, drift-diffusion model analysis suggested that free-choice performance and evidence accumulation (i.e., reduced drift rates) were degraded during initial learning-leading to weaker associations that were more easily overridden in later trial blocks (i.e., stronger bias). Together, these results suggest that in addition to attention-related functions, the ACC contributes to the ability to initiate trials and generally stay on task.

Interactive cognitive maps support flexible behavior under threat.

In social environments, survival can depend upon inferring and adapting to other agents' goal-directed behavior. However, it remains unclear how humans achieve this, despite the fact that many decisions must account for complex, dynamic agents acting according to their own goals. Here, we use a predator-prey task (total n = 510) to demonstrate that humans exploit an interactive cognitive map of the social environment to infer other agents' preferences and simulate their future behavior, providing for flexible, generalizable responses. A model-based inverse reinforcement learning model explained participants' inferences about threatening agents' preferences, with participants using this inferred knowledge to enact generalizable, model-based behavioral responses. Using tree-search planning models, we then found that behavior was best explained by a planning algorithm that incorporated simulations of the threat's goal-directed behavior. Our results indicate that humans use a cognitive map to determine other agents' preferences, facilitating generalized predictions of their behavior and effective responses.

Computationally-defined markers of uncertainty aversion predict emotional responses during a global pandemic.

Exposure to stressful life events involving threat and uncertainty often results in the development of anxiety. However, the factors that confer risk and resilience for anxiety following real world stress at a computational level remain unclear. We identified core components of uncertainty aversion moderating response to stress posed by the COVID-19 pandemic derived from computational modeling of decision making. Using both cross-sectional and longitudinal analyses, we investigated both immediate effects at the onset of the stressor, as well as medium-term changes in response to persistent stress. 479 subjects based in the United States completed a decision-making task measuring risk aversion, loss aversion, and ambiguity aversion in the early stages of the pandemic (March 2020). Self-report measures targeting threat perception, anxiety, and avoidant behavior in response to the pandemic were collected at the same time point and 8 weeks later (May 2020). Cross-sectional analyses indicated that higher risk aversion predicted higher perceived threat from the pandemic, and ambiguity aversion for guaranteed gains predicted perceived threat and pandemic-related anxiety. In longitudinal analyses, ambiguity aversion for guaranteed gains predicted greater increases in perceived infection likelihood. Together, these results suggest that individuals who have a low-level aversion toward uncertainty show stronger negative emotional reactions to both the onset and persistence of real-life stress. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Anxiety increases information-seeking in response to large changes.

Seeking information when anxious may help reduce the aversive feeling of uncertainty and guide decision-making. If information is negative or confusing, however, this may increase anxiety further. Information gathered under anxiety can thus be beneficial and/or damaging. Here, we examine whether anxiety leads to a general increase in information-seeking, or rather to changes in the type of information and/or situations in which it is sought. In two controlled laboratory studies, we show that both trait anxiety and induced anxiety lead to a selective alteration in information-seeking. In particular, anxiety did not enhance the general tendency to seek information, nor did it alter the valence of the information gathered. Rather, anxiety amplified the tendency to seek information more in response to large changes in the environment. This was true even when the cause of the anxiety was not directly related to the information sought. As anxious individuals have been shown to have problems learning in changing environments, greater information-seeking in such environments may be an adaptive compensatory mechanism.

Domain specificity versus process specificity: The "social brain" during strategic interaction.

Are social brain responses domain specific, or do domain-general but socially prevalent cognitive processes drive activity in this network? In this issue of Neuron, Konovalov et al. (2021) address this by dissociating general sociality from reactivity, one defining feature of social interactions.

How representative are neuroimaging samples? Large-scale evidence for trait anxiety differences between fMRI and behaviour-only research participants.

Over the past three decades, functional magnetic resonance imaging (fMRI) has become crucial to study how cognitive processes are implemented in the human brain. However, the question of whether participants recruited into fMRI studies differ from participants recruited into other study contexts has received little to no attention. This is particularly pertinent when effects fail to generalize across study contexts: for example, a behavioural effect discovered in a non-imaging context not replicating in a neuroimaging environment. Here, we tested the hypothesis, motivated by preliminary findings (N = 272), that fMRI participants differ from behaviour-only participants on one fundamental individual difference variable: trait anxiety. Analysing trait anxiety scores and possible confounding variables from healthy volunteers across multiple institutions (N = 3317), we found robust support for lower trait anxiety in fMRI study participants, consistent with a sampling or self-selection bias. The bias was larger in studies that relied on phone screening (compared with full in-person psychiatric screening), recruited at least partly from convenience samples (compared with community samples), and in pharmacology studies. Our findings highlight the need for surveying trait anxiety at recruitment and for appropriate screening procedures or sampling strategies to mitigate this bias.

Why and how the brain weights contributions from a mixture of experts.

It has long been suggested that human behavior reflects the contributions of multiple systems that cooperate or compete for behavioral control. Here we propose that the brain acts as a "Mixture of Experts" in which different expert systems propose strategies for action. It will be argued that the brain determines which experts should control behavior at any one moment in time by keeping track of the reliability of the predictions within each system, and by allocating control over behavior in a manner that depends on the relative reliabilities across experts. fMRI and neurostimulation studies suggest a specific contribution of the anterior prefrontal cortex in this process. Further, such a mechanism also takes into consideration the complexity of the expert, favoring simpler over more cognitively complex experts. Results from the study of different expert systems in both experiential and social learning domains hint at the possibility that this reliability-based control mechanism is domain general, exerting control over many different expert systems simultaneously in order to produce sophisticated behavior.

Economic Decisions with Ambiguous Outcome Magnitudes Vary with Low and High Stakes but Not Trait Anxiety or Depression.

Most of life's decisions involve risk and uncertainty regarding whether reward or loss will follow. Decision makers often face uncertainty not only about the likelihood of outcomes (what are the chances that I will get a raise if I ask my supervisor? What are the chances that my supervisor will be upset with me for asking?) but also the magnitude of outcomes (if I do get a raise, how large will it be? If my supervisor gets upset, how bad will the consequences be for me?). Only a few studies have investigated economic decision making with ambiguous likelihoods, and even fewer have investigated ambiguous outcome magnitudes. In the present report, we investigated the effects of ambiguous outcome magnitude, risk, and gains/losses in an economic decision-making task with low stakes (Study 1; $3.60-$5.70; N = 367) and high stakes (Study 2; $6-$48; N = 210) using a within-subjects design. We conducted computational modeling to determine individuals' preferences/aversions for ambiguous outcome magnitudes, risk, and gains/losses. We additionally investigated the association between trait anxiety and trait depression and decision-making parameters. Our results show that increasing stakes increased ambiguous gain aversion and unambiguous risk aversion but increased ambiguous sure loss preference; participants also became more averse to ambiguous sure gains relative to unambiguous risky gains. There were no significant effects of trait anxiety or trait depression on economic decision making. Our results suggest that as stakes increase, people tend to avoid uncertainty in the gain domain (especially ambiguous gains) but prefer ambiguous vs unambiguous sure losses.

Repetitive negative thinking following exposure to a natural stressor prospectively predicts altered stress responding and decision-making in the laboratory.

Repetitive negative thinking (RNT) represents a transdiagnostic risk factor for affective disorders, and stress is theorized to exacerbate this vulnerability. One mechanism by which stress may influence individual differences in psychiatric symptoms is through altered decision-making, and loss aversion in particular. The present study uses multiple methods to investigate the relationships between RNT, stress, and decision-making. We measured RNT in young adults (N = 90) recently exposed to a natural stressor, Hurricane Irma, and tested the influence of RNT on changes in affect, cortisol, and decision-making during a laboratory stress induction two months later. Post-hurricane RNT predicted greater increases in loss averse decision-making (ß = 0.30 [0.14, 0.47], p < .001; rp2 = 0.079) and negative affect (ß = 0.59 [0.37, 0.81], p < .001; rp2 = 0.319) during the early-phase response to the laboratory stressor, as well as poorer cortisol recovery (ß = 0.32, [0.10, 0.54], p = .005; rp2 = 0.095) in the late-phase stress response. Results highlight the role of loss aversion and stress in understanding RNT as an affective vulnerability factor.

A Neuro-computational Account of Arbitration between Choice Imitation and Goal Emulation during Human Observational Learning.

When individuals learn from observing the behavior of others, they deploy at least two distinct strategies. Choice imitation involves repeating other agents' previous actions, whereas emulation proceeds from inferring their goals and intentions. Despite the prevalence of observational learning in humans and other social animals, a fundamental question remains unaddressed: how does the brain decide which strategy to use in a given situation? In two fMRI studies (the second a pre-registered replication of the first), we identify a neuro-computational mechanism underlying arbitration between choice imitation and goal emulation. Computational modeling, combined with a behavioral task that dissociated the two strategies, revealed that control over behavior was adaptively and dynamically weighted toward the most reliable strategy. Emulation reliability, the model's arbitration signal, was represented in the ventrolateral prefrontal cortex, temporoparietal junction, and rostral cingulate cortex. Our replicated findings illuminate the computations by which the brain decides to imitate or emulate others.

Neural predictors of treatment response to brain stimulation and psychological therapy in depression: a double-blind randomized controlled trial.

Standard depression treatments, including antidepressant medication and cognitive behavioural therapy (CBT), are ineffective for many patients. Prefrontal transcranial direct current stimulation (tDCS) has been proposed as an alternative treatment, but has shown inconsistent efficacy for depression, and its mechanisms are poorly understood. We recruited unmedicated patients with major depressive disorder (N = 71 approached; N = 39 randomised) for a mechanistic, double-blind, randomized controlled trial consisting of eight weekly sessions of prefrontal tDCS administered to the left prefrontal cortex prior to CBT. We probed (1) whether tDCS improved the efficacy of CBT relative to sham stimulation; and (2) whether neural measures predicted clinical response. We found a modest and non-significant effect of tDCS on clinical outcome over and above CBT (active: 50%; sham: 31.6%; odds ratio: 2.16, 95% CI = 0.59-7.99), but a strong relationship, predicted a priori, between baseline activation during a working memory task in the stimulated prefrontal region and symptom improvement. Repeating our analyses of symptom outcome splitting the sample according to this biomarker revealed that tDCS was significantly superior to sham in individuals with high left prefrontal cortex activation at baseline; we also show 86% accuracy in predicting clinical response using this measure. Exploratory analyses revealed several other regions where activation at baseline was associated with subsequent response to CBT, irrespective of tDCS. This mechanistic trial revealed variable, but predictable, clinical effects of prefrontal tDCS combined with CBT for depression. We have discovered a potential explanation for this variability: individual differences in baseline activation of the region stimulated. Such a biomarker could potentially be used to pre-select patients for trials and, eventually, in the clinic.

The application of computational models to social neuroscience: promises and pitfalls.

Interactions with conspecifics are key to any social species. In order to navigate this social world, it is crucial for individuals to learn from and about others. From learning new skills by observing parents perform them to making complex collective decisions, understanding the mechanisms underlying social cognitive processes has been of considerable interest to psychologists and neuroscientists. Here, we review studies that have used computational modelling techniques, combined with neuroimaging, to shed light on how people learn and make decisions in social contexts. As opposed to standard social neuroscience methods, the computational approach allows one to directly examine where in the brain particular computations, as estimated by models of behavior, are implemented. Findings suggest that people use several strategies to learn from others: vicarious reward learning, where one learns from observing the reward outcomes of another agent; action imitation, which relies on encoding a prediction error between the expected and actual actions of the other agent; and social inference, where one learns by inferring the goals and intentions of others. These computations are implemented in distinct neural networks, which may be recruited adaptively depending on task demands, the environment and other social factors.

Valuation of knowledge and ignorance in mesolimbic reward circuitry.

The pursuit of knowledge is a basic feature of human nature. However, in domains ranging from health to finance people sometimes choose to remain ignorant. Here, we show that valence is central to the process by which the human brain evaluates the opportunity to gain information, explaining why knowledge may not always be preferred. We reveal that the mesolimbic reward circuitry selectively treats the opportunity to gain knowledge about future favorable outcomes, but not unfavorable outcomes, as if it has positive utility. This neural coding predicts participants' tendency to choose knowledge about future desirable outcomes more often than undesirable ones, and to choose ignorance about future undesirable outcomes more often than desirable ones. Strikingly, participants are willing to pay both for knowledge and ignorance as a function of the expected valence of knowledge. The orbitofrontal cortex (OFC), however, responds to the opportunity to receive knowledge over ignorance regardless of the valence of the information. Connectivity between the OFC and mesolimbic circuitry could contribute to a general preference for knowledge that is also modulated by valence. Our findings characterize the importance of valence in information seeking and its underlying neural computation. This mechanism could lead to suboptimal behavior, such as when people reject medical screenings or monitor investments more during bull than bear markets.

Enhanced Risk Aversion, But Not Loss Aversion, in Unmedicated Pathological Anxiety.

BACKGROUND: Anxiety disorders are associated with disruptions in both emotional processing and decision making. As a result, anxious individuals often make decisions that favor harm avoidance. However, this bias could be driven by enhanced aversion to uncertainty about the decision outcome (e.g., risk) or aversion to negative outcomes (e.g., loss). Distinguishing between these possibilities may provide a better cognitive understanding of anxiety disorders and hence inform treatment strategies. METHODS: To address this question, unmedicated individuals with pathological anxiety (n = 25) and matched healthy control subjects (n = 23) completed a gambling task featuring a decision between a gamble and a safe (certain) option on every trial. Choices on one type of gamble-involving weighing a potential win against a potential loss (mixed)-could be driven by both loss and risk aversion, whereas choices on the other type-featuring only wins (gain only)-were exclusively driven by risk aversion. By fitting a computational prospect theory model to participants' choices, we were able to reliably estimate risk and loss aversion and their respective contribution to gambling decisions. RESULTS: Relative to healthy control subjects, pathologically anxious participants exhibited enhanced risk aversion but equivalent levels of loss aversion. CONCLUSIONS: Individuals with pathological anxiety demonstrate clear avoidance biases in their decision making. These findings suggest that this may be driven by a reduced propensity to take risks rather than a stronger aversion to losses. This important clarification suggests that psychological interventions for anxiety should focus on reducing risk sensitivity rather than reducing sensitivity to negative outcomes per se.

Models of Affective Decision Making: How Do Feelings Predict Choice?

Intuitively, how you feel about potential outcomes will determine your decisions. Indeed, an implicit assumption in one of the most influential theories in psychology, prospect theory, is that feelings govern choice. Surprisingly, however, very little is known about the rules by which feelings are transformed into decisions. Here, we specified a computational model that used feelings to predict choices. We found that this model predicted choice better than existing value-based models, showing a unique contribution of feelings to decisions, over and above value. Similar to the value function in prospect theory, our feeling function showed diminished sensitivity to outcomes as value increased. However, loss aversion in choice was explained by an asymmetry in how feelings about losses and gains were weighted when making a decision, not by an asymmetry in the feelings themselves. The results provide new insights into how feelings are utilized to reach a decision.

Anxiety promotes memory for mood-congruent faces but does not alter loss aversion.

Pathological anxiety is associated with disrupted cognitive processing, including working memory and decision-making. In healthy individuals, experimentally-induced state anxiety or high trait anxiety often results in the deployment of adaptive harm-avoidant behaviours. However, how these processes affect cognition is largely unknown. To investigate this question, we implemented a translational within-subjects anxiety induction, threat of shock, in healthy participants reporting a wide range of trait anxiety scores. Participants completed a gambling task, embedded within an emotional working memory task, with some blocks under unpredictable threat and others safe from shock. Relative to the safe condition, threat of shock improved recall of threat-congruent (fearful) face location, especially in highly trait anxious participants. This suggests that threat boosts working memory for mood-congruent stimuli in vulnerable individuals, mirroring memory biases in clinical anxiety. By contrast, Bayesian analysis indicated that gambling decisions were better explained by models that did not include threat or treat anxiety, suggesting that: (i) higher-level executive functions are robust to these anxiety manipulations; and (ii) decreased risk-taking may be specific to pathological anxiety. These findings provide insight into the complex interactions between trait anxiety, acute state anxiety and cognition, and may help understand the cognitive mechanisms underlying adaptive anxiety.

Emotion-induced loss aversion and striatal-amygdala coupling in low-anxious individuals.

Adapting behavior to changes in the environment is a crucial ability for survival but such adaptation varies widely across individuals. Here, we asked how humans alter their economic decision-making in response to emotional cues, and whether this is related to trait anxiety. Developing an emotional decision-making task for functional magnetic resonance imaging, in which gambling decisions were preceded by emotional and non-emotional primes, we assessed emotional influences on loss aversion, the tendency to overweigh potential monetary losses relative to gains. Our behavioral results revealed that only low-anxious individuals exhibited increased loss aversion under emotional conditions. This emotional modulation of decision-making was accompanied by a corresponding emotion-elicited increase in amygdala-striatal functional connectivity, which correlated with the behavioral effect across participants. Consistent with prior reports of 'neural loss aversion', both amygdala and ventral striatum tracked losses more strongly than gains, and amygdala loss aversion signals were exaggerated by emotion, suggesting a potential role for this structure in integrating value and emotion cues. Increased loss aversion and striatal-amygdala coupling induced by emotional cues may reflect the engagement of adaptive harm-avoidance mechanisms in low-anxious individuals, possibly promoting resilience to psychopathology.