Tonic and phasic transcutaneous auricular vagus nerve stimulation (taVNS) both evoke rapid and transient pupil dilation.

BACKGROUND: Transcutaneous auricular vagus nerve stimulation (tVNS or taVNS) is a non-invasive method of electrical stimulation of the afferent pathway of the vagus nerve, suggested to drive changes in putative physiological markers of noradrenergic activity, including pupil dilation. OBJECTIVE: However, it is unknown whether different taVNS modes can map onto the phasic and tonic modes of noradrenergic activity. The effects of taVNS on pupil dilation in humans are inconsistent, largely due to differences in stimulation protocols. Here, we attempted to address these issues. METHODS: We investigated pupil dilation under phasic (1 s) and tonic (30 s) taVNS, in a pre-registered, single-blind, sham-controlled, within-subject cross-over design, in the absence of a behavioural task. RESULTS: Phasic taVNS induced a rapid increase in pupil size over baseline, significantly greater than under sham stimulation, which rapidly declined after stimulation offset. Tonic taVNS induced a similarly rapid (and larger than sham) increase in pupil size over baseline, returning to baseline within 5 s, despite the ongoing stimulation. Thus, both active and sham tonic modes closely resembled the phasic effect. There were no differences in tonic baseline pupil size, and no sustained effects of stimulation on tonic baseline pupil size. CONCLUSIONS: These results suggest that both phasic- and tonic-like taVNS under the standard stimulation parameters may modulate primarily the phasic mode of noradrenergic activity, as indexed by evoked pupil dilation, over and above somatosensory effects. This result sheds light on the temporal profile of phasic and tonic stimulation, with implications for their applicability in further research.

Stimulus awareness is necessary for both instrumental learning and instrumental responding to previously learned stimuli.

Instrumental conditioning is a crucial part of adaptive behaviour, allowing agents to selectively interact with stimuli in their environment. Recent evidence suggests that instrumental conditioning cannot proceed without stimulus awareness. However, whether accurate unconscious instrumental responding can emerge from consciously acquired knowledge of the stimulus-action-outcome contingencies is unknown. We studied this question using instrumental trace conditioning, where participants learned to make approach/avoid decisions in two within-subject modes: conscious (stimuli in plain view) and unconscious (visually masked). Both tasks were followed by an unconscious-only instrumental performance task. We show that even when the contingencies are reliably learned in the conscious mode, participants fail to act upon them in the unconscious responding task. We also replicate the previous results that no instrumental learning occurs in the unconscious mode. Consequently, the absence of stimulus awareness not only precludes instrumental conditioning, but also precludes any kind of instrumental responding to already known stimuli. This suggests that instrumental behaviour is entirely supported by conscious awareness of the world, and corroborates the proposals that consciousness may be necessary for adaptive behaviours requiring selective action.

Effort-based decision making and motivational deficits in stroke patients.

Motivational deficits in patients recovering from stroke are common and can reduce active participation in rehabilitation and thereby impede functional recovery. We investigated whether stroke patients with clinically reduced drive, initiation, and endurance during functional rehabilitative training (n = 30) display systematic alterations in effort-based decision making compared to age, sex, and severity-matched stroke patients (n = 30) whose drive appeared unaffected. Notably, the two groups did not differ in self-reported ratings of apathy and depression. However, on an effort-based decision-making task, stroke patients with clinically apparent drive impairment showed intact willingness to accept effort for reward, but were more likely to fail to execute the required effort compared to patients without apparent drive impairments. In other words, the decision behavioural assessment revealed that stroke patients that displayed reduced drive, initiation, and endurance during inpatient neurorehabilitation failed to persist in goal-directed effort production, even over very short periods. These findings indicate that reduced drive during rehabilitative therapy in post-stroke patients is not due to a diminished motivation to invest physical effort, but instead is related to a reduced persistence with effortful behaviour.

Auditory inputs modulate intrinsic neuronal timescales during sleep.

Functional magnetic resonance imaging (fMRI) studies have demonstrated that intrinsic neuronal timescales (INT) undergo modulation by external stimulation during consciousness. It remains unclear if INT keep the ability for significant stimulus-induced modulation during primary unconscious states, such as sleep. This fMRI analysis addresses this question via a dataset that comprises an awake resting-state plus rest and stimulus states during sleep. We analyzed INT measured via temporal autocorrelation supported by median frequency (MF) in the frequency-domain. Our results were replicated using a biophysical model. There were two main findings: (1) INT prolonged while MF decreased from the awake resting-state to the N2 resting-state, and (2) INT shortened while MF increased during the auditory stimulus in sleep. The biophysical model supported these results by demonstrating prolonged INT in slowed neuronal populations that simulate the sleep resting-state compared to an awake state. Conversely, under sine wave input simulating the stimulus state during sleep, the model's regions yielded shortened INT that returned to the awake resting-state level. Our results highlight that INT preserve reactivity to stimuli in states of unconsciousness like sleep, enhancing our understanding of unconscious brain dynamics and their reactivity to stimuli.

Intermediately synchronised brain states optimise trade-off between subject specificity and predictive capacity.

Functional connectivity (FC) refers to the statistical dependencies between activity of distinct brain areas. To study temporal fluctuations in FC within the duration of a functional magnetic resonance imaging (fMRI) scanning session, researchers have proposed the computation of an edge time series (ETS) and their derivatives. Evidence suggests that FC is driven by a few time points of high-amplitude co-fluctuation (HACF) in the ETS, which may also contribute disproportionately to interindividual differences. However, it remains unclear to what degree different time points actually contribute to brain-behaviour associations. Here, we systematically evaluate this question by assessing the predictive utility of FC estimates at different levels of co-fluctuation using machine learning (ML) approaches. We demonstrate that time points of lower and intermediate co-fluctuation levels provide overall highest subject specificity as well as highest predictive capacity of individual-level phenotypes.

Removal of reinforcement improves instrumental performance in humans by decreasing a general action bias rather than unmasking learnt associations.

Performance during instrumental learning is commonly believed to reflect the knowledge that has been acquired up to that point. However, recent work in rodents found that instrumental performance was enhanced during periods when reinforcement was withheld, relative to periods when reinforcement was provided. This suggests that reinforcement may mask acquired knowledge and lead to impaired performance. In the present study, we investigated whether such a beneficial effect of removing reinforcement translates to humans. Specifically, we tested whether performance during learning was improved during non-reinforced relative to reinforced task periods using signal detection theory and a computational modelling approach. To this end, 60 healthy volunteers performed a novel visual go/no-go learning task with deterministic reinforcement. To probe acquired knowledge in the absence of reinforcement, we interspersed blocks without feedback. In these non-reinforced task blocks, we found an increased d', indicative of enhanced instrumental performance. However, computational modelling showed that this improvement in performance was not due to an increased sensitivity of decision making to learnt values, but to a more cautious mode of responding, as evidenced by a reduction of a general response bias. Together with an initial tendency to act, this is sufficient to drive differential changes in hit and false alarm rates that jointly lead to an increased d'. To conclude, the improved instrumental performance in the absence of reinforcement observed in studies using asymmetrically reinforced go/no-go tasks may reflect a change in response bias rather than unmasking latent knowledge.

Motor cortical signals reflecting decision making and action preparation.

Decision making often requires accumulating evidence in favour of a particular option. When choices are expressed with a motor response, these actions are preceded by reductions in the power of oscillations in the alpha and beta range in motor cortices. For unimanual movements, these reductions are greater over the hemisphere contralateral to the response side. Such lateralizations are hypothesized to be an online index of the neural state of decisions as they develop over time of processing. In contrast, the lateralized readiness potential (LRP) is considered to selectively activate a response and appears shortly before the motor output. We investigated to what extent these neural signals reflect integration of decision evidence or more motor-related action preparation. Using two different experiments, we found that lateralization of alpha and beta power (APL and BPL, respectively) rapidly emerged after stimulus presentation, even when making an overt response was not yet possible. In contrast, we show that even after prolonged stimulus presentation, no LRP was present. Instead, the LRP emerged only after an imperative cue, prompting participants to indicate their choice. Furthermore, we could show that variations in sensory evidence strength modulate APL and BPL onset times, suggesting that integration of evidence is represented in these motor cortical signals. We conclude that APL and BPL reflect higher cognitive processes rather than pure action preparation, whereas LRP is more closely tied to motor performance. APL and BPL potentially encode decision information in motor areas serving the later preparation of overt decision output.

Reinstatement of Cortical Outcome Representations during Higher-Order Learning.

Naturalistic learning scenarios are characterized by infrequent experience of external feedback to guide behavior. Higher-order learning mechanisms like second-order conditioning (SOC) may allow stimuli that were never experienced together with reinforcement to acquire motivational value. Despite its explanatory potential for real-world learning, surprisingly little is known about the neural mechanism underlying such associative transfer of value in SOC. Here, we used multivariate cross-session, cross-modality searchlight classification on functional magnetic resonance imaging data obtained from humans during SOC. We show that visual first-order conditioned stimuli (CS) reinstate cortical patterns representing previously paired gustatory outcomes in the lateral orbitofrontal cortex (OFC). During SOC, this OFC region showed increased functional covariation with amygdala, where neural pattern similarity between second-order CS and outcomes increased from early to late stages of SOC. Our data suggest a mechanism by which motivational value is conferred to stimuli that were never paired with reinforcement.

A causal role of estradiol in human reinforcement learning.

The sex hormone estradiol is hypothesized to play a key role in human cognition, and reward processing specifically, via increased dopamine D1-receptor signalling. However, the effect of estradiol on reward processing in men has never been established. To fill this gap, we performed a double-blind placebo-controlled study in which men (N = 100) received either a single dose of estradiol (2 mg) or a placebo. Subjects performed a probabilistic reinforcement learning task where they had to choose between two options with varying reward probabilities to maximize monetary reward. Results showed that estradiol administration increased reward sensitivity compared to placebo. This effect was observed in subjects' choices, how much weight they assigned to their previous choices, and subjective reports about the reward probabilities. Furthermore, effects of estradiol were moderated by reward sensitivity, as measured through the BIS/BAS questionnaire. Using reinforcement learning models, we found that behavioral effects of estradiol were reflected in increased learning rates. These results demonstrate a causal role of estradiol within the framework of reinforcement learning, by enhancing reward sensitivity and learning. Furthermore, they provide preliminary evidence for dopamine-related genetic variants moderating the effect of estradiol on reward processing.

Not giving up: Testosterone promotes persistence against a stronger opponent.

Recent research suggests that when we lack a sense of control, we are prone to motivational failures and early quitting in competitions. Testosterone, on the other hand, is thought to boost competitiveness. Here we investigate the interaction between these factors, testing the testosterone's potential to enhance persistence in a competition against a stronger opponent, depending on experimentally manipulated perceived control. Healthy participants were administered a single dose of testosterone or placebo. They first underwent a task designed to either induce low or high perceived control and then entered a costly competition against a progressively stronger opponent that they could quit at any time. In the placebo group, men with low perceived control quitted twice as early as those with high perceived control. Testosterone countered this effect, making individuals with low control persist in the competition for as long as those with high perceived control, and did so also despite raising participants' explicit awareness of the opponents' advantage. This psychoendocrinological effect was not modulated by basal cortisol levels, CAG repeat polymorphism of the androgen receptor gene, or trait dominance. Our results provide the first causal evidence that testosterone promotes competitive persistence in humans and demonstrate that this effect depends on the psychological state elicited prior to the competition, broadening our understanding of the complex relationships between testosterone and social behaviors.

Dissociable roles of cortical excitation-inhibition balance during patch-leaving versus value-guided decisions.

In a dynamic world, it is essential to decide when to leave an exploited resource. Such patch-leaving decisions involve balancing the cost of moving against the gain expected from the alternative patch. This contrasts with value-guided decisions that typically involve maximizing reward by selecting the current best option. Patterns of neuronal activity pertaining to patch-leaving decisions have been reported in dorsal anterior cingulate cortex (dACC), whereas competition via mutual inhibition in ventromedial prefrontal cortex (vmPFC) is thought to underlie value-guided choice. Here, we show that the balance between cortical excitation and inhibition (E/I balance), measured by the ratio of GABA and glutamate concentrations, plays a dissociable role for the two kinds of decisions. Patch-leaving decision behaviour relates to E/I balance in dACC. In contrast, value-guided decision-making relates to E/I balance in vmPFC. These results support mechanistic accounts of value-guided choice and provide evidence for a role of dACC E/I balance in patch-leaving decisions.

Decisions bias future choices by modifying hippocampal associative memories.

Decision-making is guided by memories of option values. However, retrieving items from memory renders them malleable. Here, we show that merely retrieving values from memory and making a choice between options is sufficient both to induce changes to stimulus-reward associations in the hippocampus and to bias future decision-making. After allowing participants to make repeated choices between reward-conditioned stimuli, in the absence of any outcome, we observe that participants prefer stimuli they have previously chosen, and neglect previously unchosen stimuli, over otherwise identical-valued options. Using functional brain imaging, we show that decisions induce changes to hippocampal representations of stimulus-outcome associations. These changes are correlated with future decision biases. Our results indicate that choice-induced preference changes are partially driven by choice-induced modification of memory representations and suggest that merely making a choice - even without experiencing any outcomes - induces associative plasticity.

Publisher Correction: Learning relative values in the striatum induces violations of normative decision making.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Dorsolateral Prefrontal Cortex Enables Updating of Established Memories.

Updating established memories in light of new information is fundamental for memory to guide future behavior. However, little is known about the brain mechanisms by which existing memories can be updated. Here, we combined functional magnetic resonance imaging and multivariate representational similarity analysis to elucidate the neural mechanisms underlying the updating of consolidated memories. To this end, participants first learned face-city name pairs. Twenty-four hours later, while lying in the MRI scanner, participants were required to update some of these associations, but not others, and to encode entirely new pairs. Updating success was tested again 24 h later. Our results showed increased activity of the dorsolateral prefrontal cortex (dlPFC) specifically during the updating of existing associations that was significantly stronger than when simple retrieval or new encoding was required. The updating-related activity of the dlPFC and its functional connectivity with the hippocampus were directly linked to updating success. Furthermore, neural similarity for updated items was markedly higher in the dlPFC and this increase in dlPFC neural similarity distinguished individuals with high updating performance from those with low updating performance. Together, these findings suggest a key role of the dlPFC, presumably in interaction with the hippocampus, in the updating of established memories.

Learning relative values in the striatum induces violations of normative decision making.

To decide optimally between available options, organisms need to learn the values associated with these options. Reinforcement learning models offer a powerful explanation of how these values are learnt from experience. However, human choices often violate normative principles. We suggest that seemingly counterintuitive decisions may arise as a natural consequence of the learning mechanisms deployed by humans. Here, using fMRI and a novel behavioural task, we show that, when suddenly switched to novel choice contexts, participants' choices are incongruent with values learnt by standard learning algorithms. Instead, behaviour is compatible with the decisions of an agent learning how good an option is relative to an option with which it had previously been paired. Striatal activity exhibits the characteristics of a prediction error used to update such relative option values. Our data suggest that choices can be biased by a tendency to learn option values with reference to the available alternatives.

Reward-Guided Learning with and without Causal Attribution.

When an organism receives a reward, it is crucial to know which of many candidate actions caused this reward. However, recent work suggests that learning is possible even when this most fundamental assumption is not met. We used novel reward-guided learning paradigms in two fMRI studies to show that humans deploy separable learning mechanisms that operate in parallel. While behavior was dominated by precise contingent learning, it also revealed hallmarks of noncontingent learning strategies. These learning mechanisms were separable behaviorally and neurally. Lateral orbitofrontal cortex supported contingent learning and reflected contingencies between outcomes and their causal choices. Amygdala responses around reward times related to statistical patterns of learning. Time-based heuristic mechanisms were related to activity in sensorimotor corticostriatal circuitry. Our data point to the existence of several learning mechanisms in the human brain, of which only one relies on applying known rules about the causal structure of the task.

Acetylcholine mediates behavioral and neural post-error control.

Humans often commit errors when they are distracted by irrelevant information and no longer focus on what is relevant to the task at hand. Adjustments following errors are essential for optimizing goal achievement. The posterior medial frontal cortex (pMFC), a key area for monitoring errors, has been shown to trigger such post-error adjustments by modulating activity in visual cortical areas. However, the mechanisms by which pMFC controls sensory cortices are unknown. We provide evidence for a mechanism based on pMFC-induced recruitment of cholinergic projections to task-relevant sensory areas. Using fMRI in healthy volunteers, we found that error-related pMFC activity predicted subsequent adjustments in task-relevant visual brain areas. In particular, following an error, activity increased in those visual cortical areas involved in processing task-relevant stimulus features, whereas activity decreased in areas representing irrelevant, distracting features. Following treatment with the muscarinic acetylcholine receptor antagonist biperiden, activity in visual areas was no longer under control of error-related pMFC activity. This was paralleled by abolished post-error behavioral adjustments under biperiden. Our results reveal a prominent role of acetylcholine in cognitive control that has not been recognized thus far. Regaining optimal performance after errors critically depends on top-down control of perception driven by the pMFC and mediated by acetylcholine. This may explain the lack of adaptivity in conditions with reduced availability of cortical acetylcholine, such as Alzheimer's disease.

Rapid feedback processing in human nucleus accumbens and motor thalamus.

The nucleus accumbens (NAcc) and thalamus are integral parts in models of feedback processing. Deep brain stimulation (DBS) has been successfully employed to alleviate symptoms of psychiatric conditions including obsessive-compulsive disorder (OCD) and Tourette's syndrome (TS). Common target structures are the NAcc and the ventral anterior and ventro-lateral nuclei (VA/VL) of the thalamus, for OCD and TS, respectively. The feedback related negativity (FRN) is an event-related potential associated with feedback processing reflecting posterior medial frontal cortex (pMFC) activity. Here we report on three cases where we recorded scalp EEG and local field potentials (LFP) from externalized electrodes located in the NAcc or thalamus (VA/VL) while patients engaged in a modified time estimation task, known to engage feedback processing and elicit the FRN. Additionally, scalp EEG were recorded from 29 healthy participants (HP) engaged in the same task. The signal in all structures (pMFC, NAcc, and thalamus) was differently modulated by positive and negative feedback. LFP activity in the NAcc showed a biphasic time course after positive feedback during the FRN time interval. Negative feedback elicited a much weaker and later response. In the thalamus a monophasic modulation was recorded during the FRN time interval. Again, this modulation was more pronounced after positive performance feedback compared to negative feedback. In channels outside the target area no modulation was observed. The surface-FRN was reliably elicited on a group level in HP and showed no significant difference following negative feedback between patients and HP. German Clinical Trial Register: Neurocognitive specification of dysfunctions within basal ganglia-cortex loops and their therapeutic modulation by deep brain stimulation in patients with obsessive compulsive disorder and Tourette syndrome, http://www.drks.de/DRKS00005316.

Anxious individuals have difficulty learning the causal statistics of aversive environments.

Statistical regularities in the causal structure of the environment enable us to predict the probable outcomes of our actions. Environments differ in the extent to which action-outcome contingencies are stable or volatile. Difficulty in being able to use this information to optimally update outcome predictions might contribute to the decision-making difficulties seen in anxiety. We tested this using an aversive learning task manipulating environmental volatility. Human participants low in trait anxiety matched updating of their outcome predictions to the volatility of the current environment, as predicted by a Bayesian model. Individuals with high trait anxiety showed less ability to adjust updating of outcome expectancies between stable and volatile environments. This was linked to reduced sensitivity of the pupil dilatory response to volatility, potentially indicative of altered norepinephrinergic responsivity to changes in this aspect of environmental information.