Common intentional binding effects across diverse sensory modalities in touch-free voluntary actions.

The intentional binding effect refers to the phenomenon where the perceived temporal interval between a voluntary action and its sensory consequence is subjectively compressed. Prior research revealed the importance of tactile feedback from the keyboard on this effect. Here we examined the necessity of such tactile feedback by utilizing a touch-free key-press device without haptic feedback, and explored how initial/outcome sensory modalities (visual/auditory/tactile) and their consistency influence the intentional binding effect. Participants estimated three delay lengths (250, 550, or 850 ms) between the initial and outcome stimuli. Results showed that regardless of the combinations of sensory modalities between the initial and the outcome stimuli (i.e., modal consistency), the intentional binding effect was only observed in the 250 ms delay condition. This findings indicate a stable intentional binding effect both within and across sensory modalities, supporting the existence of a shared mechanism underlying the binding effect in touch-free voluntary actions.

The neural network of sensory attenuation: A neuroimaging meta-analysis.

Sensory attenuation refers to the reduction in sensory intensity resulting from self-initiated actions compared to stimuli initiated externally. A classic example is scratching oneself without feeling itchy. This phenomenon extends across various sensory modalities, including visual, auditory, somatosensory, and nociceptive stimuli. The internal forward model proposes that during voluntary actions, an efferent copy of the action command is sent out to predict sensory feedback. This predicted sensory feedback is then compared with the actual sensory feedback, leading to the suppression or reduction of sensory stimuli originating from self-initiated actions. To further elucidate the neural mechanisms underlying sensory attenuation effect, we conducted an extensive meta-analysis of functional magnetic resonance imaging (fMRI) and positron emission tomography (PET) studies. Utilizing activation likelihood estimation (ALE) analysis, our results revealed significant activations in a prominent cluster encompassing the right superior temporal gyrus (rSTG), right middle temporal gyrus (rMTG), and right insula when comparing external-generated with self-generated conditions. Additionally, significant activation was observed in the right anterior cerebellum when comparing self-generated to external-generated conditions. Further analysis using meta-analytic connectivity modeling (MACM) unveiled distinct brain networks co-activated with the rMTG and right cerebellum, respectively. Based on these findings, we propose that sensory attenuation arises from the suppression of reflexive inputs elicited by self-initiated actions through the internal forward modeling of a cerebellum-centered action prediction network, enabling the "sensory conflict detection" regions to effectively discriminate between inputs resulting from self-induced actions and those originating externally.

Protocol for predicting multivariate change of brain patterns using model-informed fMRI activations.

Investigating the spatially distributed information contained in fMRI data is essential for understanding brain functions. Here, we present a protocol to dynamically predict short-term changes in neural patterns using trial-by-trial blood-oxygen-level-dependent (BOLD) activity of a seed region. We describe steps for setting fMRI data acquisition parameters and quantification of changes in multivariate patterns. We then detail procedures for defining seed regions and identifying brain areas in which changes in multivariate patterns can be predicted by BOLD activity of the seed region. For complete details on the use and execution of this protocol, please refer to Möhring et al.1.

Prediction errors drive dynamic changes in neural patterns that guide behavior.

Learning describes the process by which our internal expectation models of the world are updated by surprising outcomes (prediction errors [PEs]) to improve predictions of future events. However, the mechanisms through which error signals dynamically influence existing neural representations are unknown. Here, we use functional magnetic resonance imaging (fMRI) in humans solving a two-step Markov decision task to investigate changes in neural activation patterns following PEs. Using a dynamic multivariate pattern analysis, we can show that PE-related fMRI responses in error-coding regions predict trial-by-trial changes in multivariate neural patterns in the orbitofrontal cortex, the precuneus, and the ventromedial prefrontal cortex (vmPFC). Importantly, the dynamics of these pattern changes in the vmPFC also predicted upcoming changes in choice strategies and thus highlight the importance of these pattern changes for behavior.

Humans depart from optimal computational models of interactive decision-making during competition under partial information.

Decision making under uncertainty in multiagent settings is of increasing interest in decision science. The degree to which human agents depart from computationally optimal solutions in socially interactive settings is generally unknown. Such understanding provides insight into how social contexts affect human interaction and the underlying contributions of Theory of Mind. In this paper, we adapt the well-known 'Tiger Problem' from artificial-agent research to human participants in solo and interactive settings. Compared to computationally optimal solutions, participants gathered less information before outcome-related decisions when competing than cooperating with others. These departures from optimality were not haphazard but showed evidence of improved performance through learning. Costly errors emerged under conditions of competition, yielding both lower rates of rewarding actions and accuracy in predicting others. Taken together, this work provides a novel approach and insights into studying human social interaction when shared information is partial.

Sex Differences and Exogenous Estrogen Influence Learning and Brain Responses to Prediction Errors.

Animal studies show marked sex differences as well as effects of estrogen (E2) in the mesocorticolimbic dopaminergic (DA) pathways, which play a critical role in reward processing and reinforcement learning and are also implicated in drug addiction. In this computational pharmacological fMRI study, we investigate the effects of both factors, sex and estrogen, on reinforcement learning and the dopaminergic system in humans; 67 male and 64 naturally cycling female volunteers, the latter in their low-hormone phase, were randomly assigned, double-blind, to take E2 or placebo. They completed a reinforcement learning task in the MRI scanner for which we have previously shown reward prediction error (RPE)-related activity to be dopaminergic. We found RPE-related brain activity to be enhanced in women compared with men and to a greater extent when E2 levels were elevated in both sexes. However, both factors, female sex and E2, slowed adaptation to RPEs (smaller learning rate). This discrepancy of larger RPE-related activity yet smaller learning rates can be explained by organizational sex differences and activational effects of circulating E2, which both affect DA release differently to DA receptor binding capacities.

Stress reduces both model-based and model-free neural computations during flexible learning.

Stressful events are thought to impair the flexible adaptation to changing environments, yet the underlying mechanisms are largely unknown. Here, we combined computational modeling and functional magnetic resonance imaging (fMRI) to elucidate the neurocomputational mechanisms underlying stress-induced deficits in flexible learning. Healthy participants underwent a stress or control manipulation before they completed, in the MRI scanner, a Markov decision task, frequently used to dissociate model-based and model-free contributions to choice, with repeated reversals of reward contingencies. Our results showed that stress attenuated the behavioral sensitivity to reversals in reward contingencies. Computational modeling further indicated that stress specifically affected the use of value computations for subsequent action selection. This reduced application of learned information on subsequent behavior was paralleled by a stress-induced reduction in inferolateral prefrontal cortex activity during model-free computations. For model-based learning, stress decreased specifically posterior, but not anterior, hippocampal activity, pointing to a functional segregation of model-based processing and its modulation by stress along the hippocampal longitudinal axis. Our findings shed light on the mechanisms underlying deficits in flexible learning under stress and indicate that, in highly dynamic environments, stress may hamper both model-based and model-free contributions to adaptive behavior.

A brain network supporting social influences in human decision-making.

Humans learn from their own trial-and-error experience and observing others. However, it remains unknown how brain circuits compute expected values when direct learning and social learning coexist in uncertain environments. Using a multiplayer reward learning paradigm with 185 participants (39 being scanned) in real time, we observed that individuals succumbed to the group when confronted with dissenting information but observing confirming information increased their confidence. Leveraging computational modeling and functional magnetic resonance imaging, we tracked direct valuation through experience and vicarious valuation through observation and their dissociable, but interacting neural representations in the ventromedial prefrontal cortex and the anterior cingulate cortex, respectively. Their functional coupling with the right temporoparietal junction representing instantaneous social information instantiated a hitherto uncharacterized social prediction error, rather than a reward prediction error, in the putamen. These findings suggest that an integrated network involving the brain's reward hub and social hub supports social influence in human decision-making.

Using reinforcement learning models in social neuroscience: frameworks, pitfalls and suggestions of best practices.

The recent years have witnessed a dramatic increase in the use of reinforcement learning (RL) models in social, cognitive and affective neuroscience. This approach, in combination with neuroimaging techniques such as functional magnetic resonance imaging, enables quantitative investigations into latent mechanistic processes. However, increased use of relatively complex computational approaches has led to potential misconceptions and imprecise interpretations. Here, we present a comprehensive framework for the examination of (social) decision-making with the simple Rescorla-Wagner RL model. We discuss common pitfalls in its application and provide practical suggestions. First, with simulation, we unpack the functional role of the learning rate and pinpoint what could easily go wrong when interpreting differences in the learning rate. Then, we discuss the inevitable collinearity between outcome and prediction error in RL models and provide suggestions of how to justify whether the observed neural activation is related to the prediction error rather than outcome valence. Finally, we suggest posterior predictive check is a crucial step after model comparison, and we articulate employing hierarchical modeling for parameter estimation. We aim to provide simple and scalable explanations and practical guidelines for employing RL models to assist both beginners and advanced users in better implementing and interpreting their model-based analyses.

Theory of mind and decision science: Towards a typology of tasks and computational models.

The ability to form a Theory of Mind (ToM), i.e., to theorize about others' mental states to explain and predict behavior in relation to attributed intentional states, constitutes a hallmark of human cognition. These abilities are multi-faceted and include a variety of different cognitive sub-functions. Here, we focus on decision processes in social contexts and review a number of experimental and computational modeling approaches in this field. We provide an overview of experimental accounts and formal computational models with respect to two dimensions: interactivity and uncertainty. Thereby, we aim at capturing the nuances of ToM functions in the context of social decision processes. We suggest there to be an increase in ToM engagement and multiplexing as social cognitive decision-making tasks become more interactive and uncertain. We propose that representing others as intentional and goal directed agents who perform consequential actions is elicited only at the edges of these two dimensions. Further, we argue that computational models of valuation and beliefs follow these dimensions to best allow researchers to effectively model sophisticated ToM-processes. Finally, we relate this typology to neuroimaging findings in neurotypical (NT) humans, studies of persons with autism spectrum (AS), and studies of nonhuman primates.

Dose-dependent effects of estrogen on prediction error related neural activity in the nucleus accumbens of healthy young women.

RATIONALE: Whereas the effect of the sex steroid 17-beta-estradiol (E2) on dopaminergic (DA) transmission in the nucleus accumbens (NAc) is well evidenced in female rats, studies in humans are inconsistent. Moreover, linear and inverted u-shaped dose response curves have been observed for E2's effects on hippocampal plasticity, but the shape of dose response curves for E2's effects on the NAc is much less characterized. OBJECTIVES: Investigation of dose response curves for E2's effects on DA-related neural activity in the human NAc. METHODS: Placebo or E2 valerate in doses of 2, 4, 6 or 12 mg was orally administered to 125 naturally cycling young women during the low-hormone menstruation phase on two consecutive days using a randomized, double-blinded design. The E2 treatment regimen induced a wide range of E2 levels, from physiological (2- and 4-mg groups; equivalent to cycle peak) to supraphysiological levels (6- and 12-mg groups; equivalent to early pregnancy). This made it possible to study different dose response functions for E2's effects on NAc activity. During E2 peak, participants performed a well-established reversal learning paradigm. We used trial-wise prediction errors (PE) estimated via a computational reinforcement learning model as a proxy for dopaminergic activity. Linear and quadratic regression analyses predicting PE-related NAc activity from salivary E2 levels were calculated. RESULTS: There was a positive linear relationship between PE-associated NAc activity and salivary E2 increases. CONCLUSIONS: The randomized, placebo-controlled elevation of E2 levels stimulates NAc activity in the human brain, likely mediated by dopaminergic processes.

Semantic Incongruency Interferes With Endogenous Attention in Cross-Modal Integration of Semantically Congruent Objects.

Efficient multisensory integration is often influenced by other cognitive processes including, but not limited to, semantic congruency and focused endogenous attention. Semantic congruency can re-allocate processing resources to the location of a congruent stimulus, while attention can prioritize the integration of multi-sensory stimuli under focus. Here, we explore the robustness of this phenomenon in the context of three stimuli, two of which are in the focus of endogenous attention. Participants completed an endogenous attention task with a stimulus compound consisting of 3 different objects: (1) a visual object (V) in the foreground, (2) an auditory object (A), and (3) a visual background scene object (B). Three groups of participants focused their attention on either the visual object and auditory sound (Group VA, n = 30), the visual object and the background (VB, n = 27), or the auditory sound and the background (AB, n = 30), and judged the semantic congruency of the objects under focus. Congruency varied systematically across all 3 stimuli: All stimuli could be semantically incongruent (e.g., V, ambulance; A, church bell; and B, swimming-pool) or all could be congruent (e.g., V, lion; A, roar; and B, savannah), or two objects could be congruent with the remaining one incongruent to the other two (e.g., V, duck; A, quack; and B, phone booth). Participants exhibited a distinct pattern of errors: when participants attended two congruent objects (e.g., group VA: V, lion; A, roar), in the presence of an unattended, incongruent third object (e.g., B, bath room) they tended to make more errors than in any other stimulus combination. Drift diffusion modeling of the behavioral data revealed a significantly smaller drift rate in two-congruent-attended condition, indicating slower evidence accumulation, which was likely due to interference from the unattended, incongruent object. Interference with evidence accumulation occurred independently of which pair of objects was in the focus of attention, which suggests that the vulnerability of congruency judgments to incongruent unattended distractors is not affected by sensory modalities. A control analysis ruled out the simple explanation of a negative response bias. These findings implicate that our perceptual system is highly sensitive to semantic incongruencies even when they are not endogenously attended.

Model-based lesion mapping of cognitive control using the Wisconsin Card Sorting Test.

The role of the frontal lobes in cognition and behavior has long been enigmatic. Over the past decade, computational models have provided a powerful approach to understanding cognition and decision-making. Here, we used a model-based approach to analyze data from a classical task used to assess frontal lobe function, the Wisconsin Card Sorting Test. We applied computational modeling and voxel-based lesion-symptom mapping in 328 patients with focal lesions, to uncover cognitive processes and neural correlates of test scores. Our results reveal that lesions in the right prefrontal cortex are associated with elevated perseverative errors and reductions in the model parameter of sensitivity to punishment. These findings indicate that the capacity to flexibly switch between task sets requires the detection of contingency changes, which are enabled by a sensitivity to punishment that reduces perseverative errors. We demonstrate the power of model-based approaches in understanding patterns of deficits on classical neuropsychological tasks.

Zen and the Art of Making a Bayesian Espresso.

In this issue of Neuron, Konovalov and Krajbich (2018) argue that a Bayesian inference is employed when learning new sequences and identify distinct brain networks that track the uncertainty of both the current state and the underlying pattern structure.

Causal role of the inferolateral prefrontal cortex in balancing goal-directed and habitual control of behavior.

Successful adaptation to complex environments depends on the balance of at least two systems: a flexible but slow goal-directed system encoding action-outcome associations and an efficient but rigid habitual system linking responses to preceding stimuli. Recent evidence suggests that the inferolateral prefrontal cortex (ilPFC), a region well known to contribute to cognitive control processes, may play a crucial role in the balance of goal-directed and habitual responding. This evidence, however, comes mainly from correlational data and whether the ilPFC is indeed causally involved in the goal-directed vs. habitual control of behavior is unclear. Here, we used neuro-navigated theta-burst stimulation (TBS) to either inhibit or enhance right ilPFC functionality before participants completed an instrumental learning task designed to probe goal-directed vs. habitual behavioral control. TBS did not affect overall learning performance. However, participants that had received inhibitory TBS were less able to adapt their behavior to altered task demands, indicating a shift from goal-directed towards more habitual control of behavior. Sham or excitatory TMS groups showed no such effect and were comparable in their performance to an unstimulated control group. Our findings indicate a causal role of the ilPFC in the balance of goal-directed vs. habitual control of behavior.

Linear and inverted U-shaped dose-response functions describe estrogen effects on hippocampal activity in young women.

In animals, 17-beta-estradiol (E2) enhances hippocampal plasticity in a dose-dependent, monotonically increasing manner, but this relationship can also exhibit an inverted U-shaped function. To investigate E2's dose-response function in the human hippocampus, we pharmacologically increased E2 levels in 125 naturally cycling women (who were in their low-hormone menstruation phase) to physiological (equivalent to menstrual cycle peak) and supraphysiological (equivalent to levels during early pregnancy) concentrations in a placebo-controlled design. Twenty-four hours after first E2 intake, we measured brain activity during encoding of neutral and negative pictures and then tested recognition memory 24 h after encoding. Here we report that E2 exhibits both a monotonically increasing relationship with hippocampal activity as well as an inverted U-shaped relationship, depending on the hippocampal region. Hippocampal activity exhibiting a U-shaped relationship inflects at supraphysiological E2 levels, suggesting that while E2 within physiological ranges stimulates hippocampal activity, supraphysiological ranges show opposite effects.

A Two-Way Street between Attention and Learning.

In an elegant model-based fMRI study, Leong et al. (2017) demonstrate how attention and learning interact to facilitate value-based decision-making. They combine computational modeling with empirical measures of attentional selection derived from eye-tracking data and multivariate pattern analyses.

Interaction of Instrumental and Goal-Directed Learning Modulates Prediction Error Representations in the Ventral Striatum.

Goal-directed and instrumental learning are both important controllers of human behavior. Learning about which stimulus event occurs in the environment and the reward associated with them allows humans to seek out the most valuable stimulus and move through the environment in a goal-directed manner. Stimulus-response associations are characteristic of instrumental learning, whereas response-outcome associations are the hallmark of goal-directed learning. Here we provide behavioral, computational, and neuroimaging results from a novel task in which stimulus-response and response-outcome associations are learned simultaneously but dominate behavior at different stages of the experiment. We found that prediction error representations in the ventral striatum depend on which type of learning dominates. Furthermore, the amygdala tracks the time-dependent weighting of stimulus-response versus response-outcome learning. Our findings suggest that the goal-directed and instrumental controllers dynamically engage the ventral striatum in representing prediction errors whenever one of them is dominating choice behavior. SIGNIFICANCE STATEMENT: Converging evidence in human neuroimaging studies has shown that the reward prediction errors are correlated with activity in the ventral striatum. Our results demonstrate that this region is simultaneously correlated with a stimulus prediction error. Furthermore, the learning system that is currently dominating behavioral choice dynamically engages the ventral striatum for computing its prediction errors. This demonstrates that the prediction error representations are highly dynamic and influenced by various experimental context. This finding points to a general role of the ventral striatum in detecting expectancy violations and encoding error signals regardless of the specific nature of the reinforcer itself.