Pupil dilation reflects effortful action invigoration in overcoming aversive Pavlovian biases.

"Pavlovian" or "motivational" biases describe the phenomenon that the valence of prospective outcomes modulates action invigoration: Reward prospect invigorates action, whereas punishment prospect suppresses it. The adaptive role of these biases in decision-making is still unclear. One idea is that they constitute a fast-and-frugal decision strategy in situations characterized by high arousal, e.g., in presence of a predator, which demand a quick response. In this pre-registered study (N = 35), we tested whether such a situation-induced via subliminally presented angry versus neutral faces-leads to increased reliance on Pavlovian biases. We measured trial-by-trial arousal by tracking pupil diameter while participants performed an orthogonalized Motivational Go/NoGo Task. Pavlovian biases were present in responses, reaction times, and even gaze, with lower gaze dispersion under aversive cues reflecting "freezing of gaze." The subliminally presented faces did not affect responses, reaction times, or pupil diameter, suggesting that the arousal manipulation was ineffective. However, pupil dilations reflected facets of bias suppression, specifically the physical (but not cognitive) effort needed to overcome aversive inhibition: Particularly strong and sustained dilations occurred when participants managed to perform Go responses to aversive cues. Conversely, no such dilations occurred when they managed to inhibit responses to Win cues. These results suggest that pupil diameter does not reflect response conflict per se nor the inhibition of prepotent responses, but specifically effortful action invigoration as needed to overcome aversive inhibition. We discuss our results in the context of the "value of work" theory of striatal dopamine.

Prefrontal signals precede striatal signals for biased credit assignment in motivational learning biases.

Actions are biased by the outcomes they can produce: Humans are more likely to show action under reward prospect, but hold back under punishment prospect. Such motivational biases derive not only from biased response selection, but also from biased learning: humans tend to attribute rewards to their own actions, but are reluctant to attribute punishments to having held back. The neural origin of these biases is unclear. Specifically, it remains open whether motivational biases arise primarily from the architecture of subcortical regions or also reflect cortical influences, the latter being typically associated with increased behavioral flexibility and control beyond stereotyped behaviors. Simultaneous EEG-fMRI allowed us to track which regions encoded biased prediction errors in which order. Biased prediction errors occurred in cortical regions (dorsal anterior and posterior cingulate cortices) before subcortical regions (striatum). These results highlight that biased learning is not a mere feature of the basal ganglia, but arises through prefrontal cortical contributions, revealing motivational biases to be a potentially flexible, sophisticated mechanism.

Goal-directed recruitment of Pavlovian biases through selective visual attention.

Prospective outcomes bias behavior in a "Pavlovian" manner: Reward prospect invigorates action, while punishment prospect suppresses it. Theories have posited Pavlovian biases as global action "priors" in unfamiliar or uncontrollable environments. However, this account fails to explain the strength of these biases-causing frequent action slips-even in well-known environments. We propose that Pavlovian control is additionally useful if flexibly recruited by instrumental control. Specifically, instrumental action plans might shape selective attention to reward/punishment information and thus the input to Pavlovian control. In two eye-tracking samples (N = 35/64), we observed that Go/NoGo action plans influenced when and for how long participants attended to reward/punishment information, which in turn biased their responses in a Pavlovian manner. Participants with stronger attentional effects showed higher performance. Thus, humans appear to align Pavlovian control with their instrumental action plans, extending its role beyond action defaults to a powerful tool ensuring robust action execution. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Effect of lysergic acid diethylamide (LSD) on reinforcement learning in humans.

BACKGROUND: The non-selective serotonin 2A (5-HT2A) receptor agonist lysergic acid diethylamide (LSD) holds promise as a treatment for some psychiatric disorders. Psychedelic drugs such as LSD have been suggested to have therapeutic actions through their effects on learning. The behavioural effects of LSD in humans, however, remain incompletely understood. Here we examined how LSD affects probabilistic reversal learning (PRL) in healthy humans. METHODS: Healthy volunteers received intravenous LSD (75 µg in 10 mL saline) or placebo (10 mL saline) in a within-subjects design and completed a PRL task. Participants had to learn through trial and error which of three stimuli was rewarded most of the time, and these contingencies switched in a reversal phase. Computational models of reinforcement learning (RL) were fitted to the behavioural data to assess how LSD affected the updating ('learning rates') and deployment of value representations ('reinforcement sensitivity') during choice, as well as 'stimulus stickiness' (choice repetition irrespective of reinforcement history). RESULTS: Raw data measures assessing sensitivity to immediate feedback ('win-stay' and 'lose-shift' probabilities) were unaffected, whereas LSD increased the impact of the strength of initial learning on perseveration. Computational modelling revealed that the most pronounced effect of LSD was the enhancement of the reward learning rate. The punishment learning rate was also elevated. Stimulus stickiness was decreased by LSD, reflecting heightened exploration. Reinforcement sensitivity differed by phase. CONCLUSIONS: Increased RL rates suggest LSD induced a state of heightened plasticity. These results indicate a potential mechanism through which revision of maladaptive associations could occur in the clinical application of LSD.

Aversive Pavlovian inhibition in adult attention-deficit/hyperactivity disorder and its restoration by mindfulness-based cognitive therapy.

Background: Control over the tendency to make or withhold responses guided by contextual Pavlovian information plays a key role in understanding impulsivity and hyperactivity. Here we set out to assess (1) the understudied relation between contextual Pavlovian inhibitory control and hyperactivity/impulsivity in adults with ADHD and (2) whether this inhibition can be enhanced by mindfulness based cognitive therapy (MBCT). Methods: Within the framework of a randomized controlled trial 50 Adult ADHD patients were assessed before and after 8 weeks of treatment as usual (TAU) with (n = 24) or without (n = 26) MBCT. We employed a well-established behavioral Pavlovian-to-instrumental transfer task that quantifies Pavlovian inhibitory control over instrumental behavior. Results: Task results revealed (1) less aversive Pavlovian inhibition in ADHD patients with clinically relevant hyperactivity/impulsivity than in those without; and (2) enhanced Pavlovian inhibition across all ADHD patients after TAU+MBCT compared with TAU. Conclusion: These findings offer new insights in the neurocognitive mechanisms of hyperactivity/impulsivity in ADHD and its treatment: We reveal a role for Pavlovian inhibitory mechanisms in understanding hyperactive/impulsive behaviors in ADHD and point toward MBCT as an intervention that might influence these mechanisms.

Cortical dopamine reduces the impact of motivational biases governing automated behaviour.

Motivations shape our behaviour: the promise of reward invigorates, while in the face of punishment, we hold back. Abnormalities of motivational processing are implicated in clinical disorders characterised by excessive habits and loss of top-down control, notably substance and behavioural addictions. Striatal and frontal dopamine have been hypothesised to play complementary roles in the respective generation and control of these motivational biases. However, while dopaminergic interventions have indeed been found to modulate motivational biases, these previous pharmacological studies used regionally non-selective pharmacological agents. Here, we tested the hypothesis that frontal dopamine controls the balance between Pavlovian, bias-driven automated responding and instrumentally learned action values. Specifically, we examined whether selective enhancement of cortical dopamine either (i) enables adaptive suppression of Pavlovian control when biases are maladaptive; or (ii) non-specifically modulates the degree of bias-driven automated responding. Healthy individuals (n = 35) received the catechol-o-methyltransferase (COMT) inhibitor tolcapone in a randomised, double-blind, placebo-controlled cross-over design, and completed a motivational Go NoGo task known to elicit motivational biases. In support of hypothesis (ii), tolcapone globally decreased motivational bias. Specifically, tolcapone improved performance on trials where the bias was unhelpful, but impaired performance in bias-congruent conditions. These results indicate a non-selective role for cortical dopamine in the regulation of motivational processes underpinning top-down control over automated behaviour. The findings have direct relevance to understanding neurobiological mechanisms underpinning addiction and obsessive-compulsive disorders, as well as highlighting a potential trans-diagnostic novel mechanism to address such symptoms.

Challenging the negative learning bias hypothesis of depression: reversal learning in a naturalistic psychiatric sample.

BACKGROUND: Classic theories posit that depression is driven by a negative learning bias. Most studies supporting this proposition used small and selected samples, excluding patients with comorbidities. However, comorbidity between psychiatric disorders occurs in up to 70% of the population. Therefore, the generalizability of the negative bias hypothesis to a naturalistic psychiatric sample as well as the specificity of the bias to depression, remain unclear. In the present study, we tested the negative learning bias hypothesis in a large naturalistic sample of psychiatric patients, including depression, anxiety, addiction, attention-deficit/hyperactivity disorder, and/or autism. First, we assessed whether the negative bias hypothesis of depression generalized to a heterogeneous (and hence more naturalistic) depression sample compared with controls. Second, we assessed whether negative bias extends to other psychiatric disorders. Third, we adopted a dimensional approach, by using symptom severity as a way to assess associations across the sample. METHODS: We administered a probabilistic reversal learning task to 217 patients and 81 healthy controls. According to the negative bias hypothesis, participants with depression should exhibit enhanced learning and flexibility based on punishment v. reward. We combined analyses of traditional measures with more sensitive computational modeling. RESULTS: In contrast to previous findings, this sample of depressed patients with psychiatric comorbidities did not show a negative learning bias. CONCLUSIONS: These results speak against the generalizability of the negative learning bias hypothesis to depressed patients with comorbidities. This study highlights the importance of investigating unselected samples of psychiatric patients, which represent the vast majority of the psychiatric population.

Striatal BOLD and Midfrontal Theta Power Express Motivation for Action.

Action selection is biased by the valence of anticipated outcomes. To assess mechanisms by which these motivational biases are expressed and controlled, we measured simultaneous EEG-fMRI during a motivational Go/NoGo learning task (N = 36), leveraging the temporal resolution of EEG and subcortical access of fMRI. VmPFC BOLD encoded cue valence, importantly predicting trial-by-trial valence-driven response speed differences and EEG theta power around cue onset. In contrast, striatal BOLD encoded selection of active Go responses and correlated with theta power around response time. Within trials, theta power ramped in the fashion of an evidence accumulation signal for the value of making a "Go" response, capturing the faster responding to reward cues. Our findings reveal a dual nature of midfrontal theta power, with early components reflecting the vmPFC contribution to motivational biases, and late components reflecting their striatal translation into behavior, in line with influential recent "value of work" theories of striatal processing.

Effects of methylphenidate on reinforcement learning depend on working memory capacity.

RATIONALE: Brain catecholamines have long been implicated in reinforcement learning, exemplified by catecholamine drug and genetic effects on probabilistic reversal learning. However, the mechanisms underlying such effects are unclear. OBJECTIVES AND METHODS: Here we investigated effects of an acute catecholamine challenge with methylphenidate (20 mg, oral) on a novel probabilistic reversal learning paradigm in a within-subject, double-blind randomised design. The paradigm was designed to disentangle effects on punishment avoidance from effects on reward perseveration. Given the known large individual variability in methylphenidate's effects, we stratified our effects by working memory capacity and trait impulsivity, putatively modulating the effects of methylphenidate, in a large sample (n = 102) of healthy volunteers. RESULTS: Contrary to our prediction, methylphenidate did not alter performance in the reversal phase of the task. Our key finding is that methylphenidate altered learning of choice-outcome contingencies in a manner that depended on individual variability in working memory span. Specifically, methylphenidate improved performance by adaptively reducing the effective learning rate in participants with higher working memory capacity. CONCLUSIONS: This finding emphasises the important role of working memory in reinforcement learning, as reported in influential recent computational modelling and behavioural work, and highlights the dependence of this interplay on catecholaminergic function.

Effects of dopamine on reinforcement learning in Parkinson's disease depend on motor phenotype.

Parkinson's disease is clinically defined by bradykinesia, along with rigidity and tremor. However, the severity of these motor signs is greatly variable between individuals, particularly the presence or absence of tremor. This variability in tremor relates to variation in cognitive/motivational impairment, as well as the spatial distribution of neurodegeneration in the midbrain and dopamine depletion in the striatum. Here we ask whether interindividual heterogeneity in tremor symptoms could account for the puzzlingly large variability in the effects of dopaminergic medication on reinforcement learning, a fundamental cognitive function known to rely on dopamine. Given that tremor-dominant and non-tremor Parkinson's disease patients have different dopaminergic phenotypes, we hypothesized that effects of dopaminergic medication on reinforcement learning differ between tremor-dominant and non-tremor patients. Forty-three tremor-dominant and 20 non-tremor patients with Parkinson's disease were recruited to be tested both OFF and ON dopaminergic medication (200/50 mg levodopa-benserazide), while 22 age-matched control subjects were recruited to be tested twice OFF medication. Participants performed a reinforcement learning task designed to dissociate effects on learning rate from effects on motivational choice (i.e. the tendency to 'Go/NoGo' in the face of reward/threat of punishment). In non-tremor patients, dopaminergic medication improved reward-based choice, replicating previous studies. In contrast, in tremor-dominant patients, dopaminergic medication improved learning from punishment. Formal modelling showed divergent computational effects of dopaminergic medication as a function of Parkinson's disease motor phenotype, with a modulation of motivational choice bias and learning rate in non-tremor and tremor patients, respectively. This finding establishes a novel cognitive/motivational difference between tremor and non-tremor Parkinson's disease patients, and highlights the importance of considering motor phenotype in future work.

Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex.

RATIONALE: During value-based decision-making, organisms make choices on the basis of reward expectations, which have been formed during prior action-outcome learning. Although it is known that neuronal manipulations of different subregions of the rat prefrontal cortex (PFC) have qualitatively different effects on behavioral tasks involving value-based decision-making, it is unclear how these regions contribute to the underlying component processes. OBJECTIVES: Assessing how different regions of the rodent PFC contribute to component processes of value-based decision-making behavior, including reward (or positive feedback) learning, punishment (or negative feedback) learning, response persistence, and exploration versus exploitation. METHODS: We performed behavioral modeling of data of rats in a probabilistic reversal learning task after pharmacological inactivation of five PFC subregions, to assess how inactivation of these different regions affected the structure of responding of animals in the task. RESULTS: Our results show reductions in reward and punishment learning after PFC subregion inactivation. The prelimbic, infralimbic, lateral orbital, and medial orbital PFC particularly contributed to punishment learning, and the prelimbic and lateral orbital PFC to reward learning. In addition, response persistence depended on the infralimbic and medial orbital PFC. As a result, pharmacological inactivation of the infralimbic and lateral orbitofrontal cortex reduced the number of reversals achieved, whereas inactivation of the prelimbic and medial orbitofrontal cortex decreased the number of rewards obtained. Finally, using simulated data, we explain discrepancies with a previous study and demonstrate complex, interacting relationships between conventional measures of probabilistic reversal learning performance, such as win-stay/lose-switch behavior, and component processes of value-based decision-making. CONCLUSIONS: Together, our data suggest that distinct components of value-based learning and decision-making are generated in medial and orbital PFC regions, displaying functional specialization and overlap, with a prominent role of large parts of the PFC in negative feedback processing.

GABAergic changes in the thalamocortical circuit in Parkinson's disease.

Parkinson's disease is characterized by bradykinesia, rigidity, and tremor. These symptoms have been related to an increased gamma-aminobutyric acid (GABA)ergic inhibitory drive from globus pallidus onto the thalamus. However, in vivo empirical evidence for the role of GABA in Parkinson's disease is limited. Some discrepancies in the literature may be explained by the presence or absence of tremor. Specifically, recent functional magnetic resonance imaging (fMRI) findings suggest that Parkinson's tremor is associated with reduced, dopamine-dependent thalamic inhibition. Here, we tested the hypothesis that GABA in the thalamocortical motor circuit is increased in Parkinson's disease, and we explored differences between clinical phenotypes. We included 60 Parkinson patients with dopamine-resistant tremor (n = 17), dopamine-responsive tremor (n = 23), or no tremor (n = 20), and healthy controls (n = 22). Using magnetic resonance spectroscopy, we measured GABA-to-total-creatine ratio in motor cortex, thalamus, and a control region (visual cortex) on two separate days (ON and OFF dopaminergic medication). GABA levels were unaltered by Parkinson's disease, clinical phenotype, or medication. However, motor cortex GABA levels were inversely correlated with disease severity, particularly rigidity and tremor, both ON and OFF medication. We conclude that cortical GABA plays a beneficial rather than a detrimental role in Parkinson's disease, and that GABA depletion may contribute to increased motor symptom expression.

Suicidal thoughts and behaviors are associated with an increased decision-making bias for active responses to escape aversive states.

Suicide is a leading cause of death worldwide. Despite decades of clinical and theoretical accounts that suggest that suicidal thoughts and behaviors are efforts to escape painful emotions, little prior research has examined decision making involved in escaping aversive states. We compared the performance of 85 suicidal participants to 44 nonsuicidal psychiatric patients on a novel reinforcement learning task with choices to make either active (i.e., "go") or passive responses (i.e., "no-go") to either escape or avoid an aversive stimulus. We used a computational cognitive model to isolate decision-making biases. We hypothesized that suicidal participants would exhibit a relatively elevated bias for making active responses to escape an aversive state and would show worse performance when escape required a passive response (i.e., "doing nothing" to escape). Our hypotheses were supported: The computational model revealed that suicidal participants exhibited a higher bias for an active response to escape compared with nonsuicidal psychiatric controls, suggesting that this finding was not just the result of the presence of psychopathology. The bias parameter also accounted for unique variance in predicting group status among several constructs previously related to suicidal thoughts and behaviors. This study provides a new method for testing escape decision making and does so using a computational cognitive model, allowing us to precisely index processes underlying suicidal and related behaviors. Future research examining escape decision making from a computational perspective could help link neural processes or environmental stressors to suicidal thoughts or behaviors. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Catecholaminergic modulation of the avoidance of cognitive control.

The catecholamines have long been associated with cognitive control and value-based decision-making. More recently, we proposed that the catecholamines might modulate value-based decision-making about whether or not to engage in cognitive control. We test this hypothesis by assessing effects of a catecholamine challenge in a large sample of young, healthy adults (n = 100) on the avoidance of a cognitively demanding control process: task switching. Prolonging catecholamine transmission by blocking reuptake with methylphenidate altered the avoidance, but not the execution of cognitive control. Crucially, these effects could be isolated by taking into account individual differences in trait impulsivity, so that participants with higher trait impulsivity became more avoidant of cognitive control, despite faster task performance. One implication of these findings is that performance-enhancing effects of methylphenidate may be accompanied by an undermining effect on the willingness to exert cognitive control. Taken together, these findings integrate hitherto segregated literatures on catecholamines' roles in value-based learning/choice and cognitive control. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Frontal network dynamics reflect neurocomputational mechanisms for reducing maladaptive biases in motivated action.

Motivation exerts control over behavior by eliciting Pavlovian responses, which can either match or conflict with instrumental action. We can overcome maladaptive motivational influences putatively through frontal cognitive control. However, the neurocomputational mechanisms subserving this control are unclear; does control entail up-regulating instrumental systems, down-regulating Pavlovian systems, or both? We combined electroencephalography (EEG) recordings with a motivational Go/NoGo learning task (N = 34), in which multiple Go options enabled us to disentangle selective action learning from nonselective Pavlovian responses. Midfrontal theta-band (4 Hz-8 Hz) activity covaried with the level of Pavlovian conflict and was associated with reduced Pavlovian biases rather than reduced instrumental learning biases. Motor and lateral prefrontal regions synchronized to the midfrontal cortex, and these network dynamics predicted the reduction of Pavlovian biases over and above local, midfrontal theta activity. This work links midfrontal processing to detecting Pavlovian conflict and highlights the importance of network processing in reducing the impact of maladaptive, Pavlovian biases.

A neuronal mechanism underlying decision-making deficits during hyperdopaminergic states.

Hyperdopaminergic states in mental disorders are associated with disruptive deficits in decision making. However, the precise contribution of topographically distinct mesencephalic dopamine pathways to decision-making processes remains elusive. Here we show, using a multidisciplinary approach, how hyperactivity of ascending projections from the ventral tegmental area (VTA) contributes to impaired flexible decision making in rats. Activation of the VTA-nucleus accumbens pathway leads to insensitivity to loss and punishment due to impaired processing of negative reward prediction errors. In contrast, activation of the VTA-prefrontal cortex pathway promotes risky decision making without affecting the ability to choose the economically most beneficial option. Together, these findings show how malfunction of ascending VTA projections affects value-based decision making, suggesting a potential mechanism through which increased forebrain dopamine signaling leads to aberrant behavior, as is seen in substance abuse, mania, and after dopamine replacement therapy in Parkinson's disease.

Pavlovian Control of Escape and Avoidance.

To survive in complex environments, animals need to have mechanisms to select effective actions quickly, with minimal computational costs. As perhaps the computationally most parsimonious of these systems, Pavlovian control accomplishes this by hardwiring specific stereotyped responses to certain classes of stimuli. It is well documented that appetitive cues initiate a Pavlovian bias toward vigorous approach; however, Pavlovian responses to aversive stimuli are less well understood. Gaining a deeper understanding of aversive Pavlovian responses, such as active avoidance, is important given the critical role these behaviors play in several psychiatric conditions. The goal of the current study was to establish a behavioral and computational framework to examine aversive Pavlovian responses (activation vs. inhibition) depending on the proximity of an aversive state (escape vs. avoidance). We introduce a novel task in which participants are exposed to primary aversive (noise) stimuli and characterized behavior using a novel generative computational model. This model combines reinforcement learning and drift-diffusion models so as to capture effects of invigoration/inhibition in both explicit choice behavior as well as changes in RT. Choice and RT results both suggest that escape is associated with a bias for vigorous action, whereas avoidance is associated with behavioral inhibition. These results lay a foundation for future work seeking insights into typical and atypical aversive Pavlovian responses involved in psychiatric disorders, allowing us to quantify both implicit and explicit indices of vigorous choice behavior in the context of aversion.

Dopamine controls Parkinson's tremor by inhibiting the cerebellar thalamus.

Parkinson's resting tremor is related to altered cerebral activity in the basal ganglia and the cerebello-thalamo-cortical circuit. Although Parkinson's disease is characterized by dopamine depletion in the basal ganglia, the dopaminergic basis of resting tremor remains unclear: dopaminergic medication reduces tremor in some patients, but many patients have a dopamine-resistant tremor. Using pharmacological functional magnetic resonance imaging, we test how a dopaminergic intervention influences the cerebral circuit involved in Parkinson's tremor. From a sample of 40 patients with Parkinson's disease, we selected 15 patients with a clearly tremor-dominant phenotype. We compared tremor-related activity and effective connectivity (using combined electromyography-functional magnetic resonance imaging) on two occasions: ON and OFF dopaminergic medication. Building on a recently developed cerebral model of Parkinson's tremor, we tested the effect of dopamine on cerebral activity associated with the onset of tremor episodes (in the basal ganglia) and with tremor amplitude (in the cerebello-thalamo-cortical circuit). Dopaminergic medication reduced clinical resting tremor scores (mean 28%, range -12 to 68%). Furthermore, dopaminergic medication reduced tremor onset-related activity in the globus pallidus and tremor amplitude-related activity in the thalamic ventral intermediate nucleus. Network analyses using dynamic causal modelling showed that dopamine directly increased self-inhibition of the ventral intermediate nucleus, rather than indirectly influencing the cerebello-thalamo-cortical circuit through the basal ganglia. Crucially, the magnitude of thalamic self-inhibition predicted the clinical dopamine response of tremor. Dopamine reduces resting tremor by potentiating inhibitory mechanisms in a cerebellar nucleus of the thalamus (ventral intermediate nucleus). This suggests that altered dopaminergic projections to the cerebello-thalamo-cortical circuit have a role in Parkinson's tremor.aww331media15307619934001.