Recording neural reward signals in a naturalistic operant task using mobile-EEG and augmented reality.

The electrophysiological response to rewards recorded during laboratory tasks has been well documented, yet little is known about the neural response patterns in a more naturalistic setting. Here, we combined a mobile-EEG system with an augmented reality headset to record event-related brain potentials (ERP) while participants engaged in a naturalistic operant task to find rewards. Twenty-five participants were asked to navigate towards a west or east goal location marked by floating orbs, and once participants reached the goal location, the orb would then signify a reward (5 cents) or no-reward (0 cents) outcome. Following the outcome, participants returned to a start location marked by floating purple rings, and once standing in the middle, a 3 second counter would signal the start of the next trial, for a total of 200 trials. Consistent with previous research, reward feedback evoked the reward positivity, an ERP component believed to index the sensitivity of the anterior cingulate cortex to reward prediction error signals. The reward positivity peaked around 230 ms post-feedback with a maximal at channel FCz (M = -0.695µV, ± .23) and was significantly different than zero (p< 0.01). Participants took approximately 3.38 seconds to reach the goal-location and exhibited a general lose-shift (68.3% ±3.5) response strategy and post-error slowing. Overall, these novel findings provide support for the idea that combining mobile-EEG with augmented reality technology is a feasible solution to enhance the ecological validity of human electrophysiological studies of goal-directed behavior and a step towards a new era of human cognitive neuroscience research that blurs the line between laboratory and reality.Significant Statement Building on decades of experimental, computational, and theoretical analyses of reinforcement learning in animal and humans, the present study reveals for the first time that scalp-recorded electrophysiological signals associated with the anterior cingulate cortex sensitivity to reward prediction error signals is dynamically modulated by rewards in humans freely navigating a more realistic environment, and that participants performed the task in accordance with reinforcement learning theory.

Oscillatory correlates of threat imminence during virtual navigation.

The Predatory Imminence Continuum Theory proposes that defensive behaviors depend on the proximity of a threat. While the neural mechanisms underlying this proposal are well studied in animal models, it remains poorly understood in humans. To address this issue, we recorded EEG from 24 (15 female) young adults engaged in a first-person virtual reality Risk-Reward interaction task. On each trial, participants were placed in a virtual room and presented with either a threat or reward conditioned stimulus (CS) in the same room location (proximal) or different room location (distal). Behaviorally, all participants learned to avoid the threat-CS, with most using the optimal behavior to actively avoid the proximal threat-CS (88% accuracy) and passively avoid the distal threat-CS (69% accuracy). Similarly, participants learned to actively approach the distal reward-CS (82% accuracy) and to remain passive to the proximal reward-CS (72% accuracy). At an electrophysiological level, we observed a general increase in theta power (4-8 Hz) over the right posterior channel P8 across all conditions, with the proximal threat-CS evoking the largest theta response. By contrast, distal cues induced two bursts of gamma (30-60 Hz) power over midline-parietal channel Pz (200 msec post-cue) and right frontal channel Fp2 (300 msec post-cue). Interestingly, the first burst of gamma power was sensitive to the distal threat-CS and the second burst at channel Fp2 was sensitive to the distal reward-CS. Together, these findings demonstrate that oscillatory processes differentiate between the spatial proximity information during threat and reward encoding, likely optimizing the selection of the appropriate behavioral response.

Recording neural reward signals in the real-world using mobile-EEG and augmented reality.

The electrophysiological response to rewards recorded during laboratory-based tasks has been well documented over the past two decades, yet little is known about the neural response patterns in 'real-world' settings. To address this issue, we combined a mobile-EEG system with an augmented reality headset (which blends high definition "holograms" within the real-world) to record event-related brain potentials (ERP) while participants navigated an operant chamber to find rewards. 25 participants (age = 18-43, Male=6, Female=19) were asked to choose between two floating holograms marking a west or east goal-location in a large room, and once participants reached the goal location, the hologram would turn into a reward (5 cents) or no-reward (0 cents) cue. Following the feedback cue, participants were required to return to a hologram marking the start location, and once standing in it, a 3 second counter hologram would initiate the next trial. This sequence was repeated until participants completed 200 trials. Consistent with previous research, reward feedback evoked the reward positivity, an ERP component believed to index the sensitivity of the anterior cingulate cortex to reward prediction error signals. The reward positivity peaked around 235ms post-feedback with a maximal at channel FCz (M=-2.60µV, SD=1.73µV) and was significantly different than zero (p < 0.01). At a behavioral level, participants took approximately 3.38 seconds to reach the goal-location and exhibited a general lose-shift (68.3% ± 3.5) response strategy and were slightly slower to return to the start location following negative feedback (2.43 sec) compared to positive feedback (2.38 sec), evidence of post-error slowing. Overall, these findings provide the first evidence that combining mobile-EEG with augmented reality technology is a feasible solution to enhance the ecological validity of human electrophysiological studies of goal-directed behavior and a step towards a new era of human cognitive neuroscience research that blurs the line between laboratory and reality.

Prefrontal transcranial magnetic stimulation boosts response vigour during reinforcement learning in healthy adults.

A 10-Hz repetitive transcranial magnetic stimulation to the left dorsal lateral prefrontal cortex has been shown to increase dopaminergic activity in the dorsal striatum, a region strongly implicated in reinforcement learning. However, the behavioural influence of this effect remains largely unknown. We tested the causal effects of 10-Hz stimulation on behavioural and computational characteristics of reinforcement learning. A total of 40 healthy individuals were randomized into active and sham (placebo) stimulation groups. Each participant underwent one stimulation session (1500 pulses) in which stimulation was applied over the left dorsal lateral prefrontal cortex using a robotic arm. Participants then completed a reinforcement learning task sensitive to striatal dopamine functioning. Participants' choices were modelled using a reinforcement learning model (Q-learning) that calculates separate learning rates associated with positive and negative reward prediction errors. Subjects receiving active stimulation exhibited increased reward rate (number of correct responses per second of task activity) compared with those in sham. Computationally, although no group differences were observed, the active group displayed a higher learning rate for correct trials (αG) compared with incorrect trials (αL). Finally, when tested with novel pairs of stimuli, the active group displayed extremely fast reaction times, and a trend towards a higher reward rate. This study provided specific behavioural and computational accounts of altered striatal-mediated behaviour, particularly response vigour, induced by a proposed increase of dopamine activity by 10-Hz stimulation to the left dorsal lateral prefrontal cortex. Together, these findings bolster the use of repetitive transcranial magnetic stimulation to target neurocognitive disturbances attributed to the dysregulation of dopaminergic-striatal circuits.

Scalp recorded theta activity is modulated by reward, direction, and speed during virtual navigation in freely moving humans.

Theta oscillations (~ 4-12 Hz) are dynamically modulated by speed and direction in freely moving animals. However, due to the paucity of electrophysiological recordings of freely moving humans, this mechanism remains poorly understood. Here, we combined mobile-EEG with fully immersive virtual-reality to investigate theta dynamics in 22 healthy adults (aged 18-29 years old) freely navigating a T-maze to find rewards. Our results revealed three dynamic periods of theta modulation: (1) theta power increases coincided with the participants' decision-making period; (2) theta power increased for fast and leftward trials as subjects approached the goal location; and (3) feedback onset evoked two phase-locked theta bursts over the right temporal and frontal-midline channels. These results suggest that recording scalp EEG in freely moving humans navigating a simple virtual T-maze can be utilized as a powerful translational model by which to map theta dynamics during "real-life" goal-directed behavior in both health and disease.

Recovery of reward function in problematic substance users using a combination of robotics, electrophysiology, and TMS.

BACKGROUND: Theoretical and empirical work suggest that addictive drugs potentiate dopaminergic reinforcement learning signals and disrupt the reward function of its neural targets, including the anterior midcingulate cortex (aMCC) and the basal ganglia. Here, we aim to use prefrontal 10-Hz TMS to enhance aMCC reward activity and reward learning by the basal ganglia in problematic substance users. METHODS: 22 problematic substance users were randomized into an Active and SHAM (coil flipped) TMS group. We recorded the reward positivity-an electrophysiological signal believed to index sensitivity of the aMCC to rewards-while participants engaged in 4 blocks (100 trials per block) of a reward-based choice task. A robotic arm positioned a TMS coil over a prefrontal cortex target, and 50 pulses were delivered at 10-Hz before every 10 trials of blocks 2-4 (1500 pulses, 400 trials). Participants then completed a decision-making task that is diagnostic of striatal dopamine dysfunction. RESULTS: The present study revealed three main findings. First, both groups failed to elicit a reward positivity during the first two task blocks. Second, applying robot-assisted TMS enhanced the amplitude of the reward positivity in the Active group, but not the SHAM group, across the last two task blocks. Third, the Active group performed relatively better at reward-based learning than the SHAM group. CONCLUSION: These results demonstrate that 10-Hz TMS is successful in modulating the reward function of the aMCC and basal ganglia in problematic substance users, which may have utility in the treatment of reward-related neural dysfunction commonly associated with substance use disorders.

Beyond the Motor Cortex: Theta Burst Stimulation of the Anterior Midcingulate Cortex.

BACKGROUND: While the facilitatory and inhibitory effects of intermittent theta burst stimulation (iTBS) and continuous TBS (cTBS) protocols have been well documented on motor physiology, the action of TBS protocols on prefrontal functioning remain unclear. Here we asked whether iTBS or cTBS can differentially modulate reward-related signaling in the anterior midcingulate cortex (aMCC). METHODS: Across 2 experiments, we used a robot-assisted transcranial magnetic stimulation system, combined with electroencephalogram recordings, to investigate the aftereffects of prefrontal iTBS and cTBS on the reward positivity, an electrophysiological signal believed to index sensitivity of the aMCC to rewards. Twenty adults (age, 18-28 years) participated in experiment 1 in which we used a scalp landmark for TBS targeting, and 14 adults (age, 18-28 years) participated in experiment 2, in which we aimed to increase TBS effectiveness by utilizing cortical thickness maps to select individualized dorsal lateral prefrontal cortex targets. RESULTS: We demonstrated that prefrontal iTBS suppressed reward-related signaling in the aMCC (reduction in reward positivity) and caused a decrease in postfeedback switch choices. cTBS displayed no effect. We replicated and strengthened this effect on the reward positivity by targeting dorsal lateral prefrontal cortex regions displaying maximal cortical thickness. CONCLUSIONS: While these results are inconsistent with reported TBS effects on motor cortex, the present findings offer a novel transcranial magnetic stimulation targeting approach and normative insights into the magnitude and time course of TBS-induced changes in aMCC excitability. By modulating how the aMCC links value to goal-directed behavior, this research opens an exciting new era of investigative possibilities in the understanding of aMCC function and treatment of aMCC dysfunction.

Smoking Decisions: Altered Reinforcement Learning Signals Induced by Nicotine State.

INTRODUCTION: Alterations in dopamine signaling play a key role in reinforcement learning and nicotine addiction, but the relationship between these two processes has not been well characterized. We investigated this relationship in young adult smokers using a combination of behavioral and computational measures of reinforcement learning. METHODS: We asked moderately dependent smokers to engage in a reinforcement learning task three times: smoking as usual, smoking abstinence, and cigarette consumption. Participants' trial-to-trial training choices were modeled using a reinforcement learning model that calculates separate learning rates associated with positive and negative prediction errors. RESULTS: We found that learning from positive prediction error signals is reduced during smoking abstinence and enhanced following cigarette consumption. By contrast, learning from negative prediction error signals was enhanced during smoking abstinence and reduced following cigarette consumption. Finally, when tested with novel pairs of stimuli, participants were relatively better at selecting the positive feedback predicting stimuli than avoiding the negative feedback predicting stimuli during the smoking as usual session, a pattern that reversed following cigarette consumption. CONCLUSIONS: These findings provide a specific computational account of altered reinforcement learning induced by smoking state (abstinence and consumption) and may represent a unique target for treatment of nicotine addiction. IMPLICATIONS: This study illustrates the potential of computational psychiatry for understanding reinforcement learning deficits associated with substance use disorders in general and nicotine addiction in particular. We found that learning from positive prediction error signals is reduced during smoking abstinence and enhanced following cigarette consumption. By contrast, learning from negative prediction error signals was enhanced during smoking abstinence and reduced following cigarette consumption. By highlighting important computational differences between three states of smoking, these findings hold out promise for integrating experimental, computational, and theoretical analyses of decision-making function together with research on addiction-related disorders.

Social context effects on error-related brain activity are dependent on interpersonal and achievement-related traits.

Brain correlates of performance monitoring, such as the Error-Related Negativity (ERN), are considerably influenced by situational factors. For instance, errors committed during social interaction typically elicit enhanced ERNs. While individual differences in ERN magnitude have been implicated in a wide variety of psychopathologies, it remains unclear how individual dispositions may interact with situational incentives to influence performance monitoring. Here, we analysed how interpersonal (Affiliation) and achievement-related (Agency) traits moderated the effects of interpersonal competition and interpersonal cooperation on the ERN. For this purpose, electroencephalography was collected from 78 participants while they performed a Flanker Task either in a competitive or in a cooperative social context (i.e., between-subjects design). We found that competition predicted enhanced error-related activity patterns compared to cooperation. Furthermore, participants who scored high in Affiliation elicited enhanced error-related activity. Conversely, high Agency scores were associated with reduced error-related activity, but this was only observed in the competitive context. These results indicate that the brain's response to error commission is not only sensitive to social incentives. Rather, the activity of the evaluative system that produces error signals appears to be crucially determined by the personal relevance of the incentives present in the context in which performance is evaluated.

Modulation of orbitofrontal-striatal reward activity by dopaminergic functional polymorphisms contributes to a predisposition to alcohol misuse in early adolescence.

BACKGROUND: Abnormalities in reward circuit function are considered a core feature of addiction. Yet, it is still largely unknown whether these abnormalities stem from chronic drug use, a genetic predisposition, or both. METHODS: In the present study, we investigated this issue using a large sample of adolescent children by applying structural equation modeling to examine the effects of several dopaminergic polymorphisms of the D1 and D2 receptor type on the reward function of the ventral striatum (VS) and orbital frontal cortex (OFC), and whether this relationship predicted the propensity to engage in early alcohol misuse behaviors at 14 years of age and again at 16 years of age. RESULTS: The results demonstrated a regional specificity with which the functional polymorphism rs686 of the D1 dopamine receptor (DRD1) gene and Taq1A of the ANKK1 gene influenced medial and lateral OFC activation during reward anticipation, respectively. Importantly, our path model revealed a significant indirect relationship between the rs686 of the DRD1 gene and early onset of alcohol misuse through a medial OFC × VS interaction. CONCLUSIONS: These findings highlight the role of D1 and D2 in adjusting reward-related activations within the mesocorticolimbic circuitry, as well as in the susceptibility to early onset of alcohol misuse.

Neurobehavioral correlates of obesity are largely heritable.

Recent molecular genetic studies have shown that the majority of genes associated with obesity are expressed in the central nervous system. Obesity has also been associated with neurobehavioral factors such as brain morphology, cognitive performance, and personality. Here, we tested whether these neurobehavioral factors were associated with the heritable variance in obesity measured by body mass index (BMI) in the Human Connectome Project (n = 895 siblings). Phenotypically, cortical thickness findings supported the "right brain hypothesis" for obesity. Namely, increased BMI is associated with decreased cortical thickness in right frontal lobe and increased thickness in the left frontal lobe, notably in lateral prefrontal cortex. In addition, lower thickness and volume in entorhinal-parahippocampal structures and increased thickness in parietal-occipital structures in participants with higher BMI supported the role of visuospatial function in obesity. Brain morphometry results were supported by cognitive tests, which outlined a negative association between BMI and visuospatial function, verbal episodic memory, impulsivity, and cognitive flexibility. Personality-BMI correlations were inconsistent. We then aggregated the effects for each neurobehavioral factor for a behavioral genetics analysis and estimated each factor's genetic overlap with BMI. Cognitive test scores and brain morphometry had 0.25-0.45 genetic correlations with BMI, and the phenotypic correlations with BMI were 77-89% explained by genetic factors. Neurobehavioral factors also had some genetic overlap with each other. In summary, obesity as measured by BMI has considerable genetic overlap with brain and cognitive measures. This supports the theory that obesity is inherited via brain function and may inform intervention strategies.

Reversing the Atypical Valuation of Drug and Nondrug Rewards in Smokers Using Multimodal Neuroimaging.

BACKGROUND: Chronic substance use can disrupt the reward function of the anterior cingulate cortex (ACC), biasing the ACC to favor goal-directed behaviors that converge on drug use. Here we used multimodal neuroimaging methods to ask whether modulating reward-related signaling in the ACC can reverse the atypical valuation of nondrug and drug rewards in abstinent smokers. METHODS: We first recorded functional magnetic resonance imaging data from 20 moderately dependent cigarette smokers (mean age = 25 years; no history of neuropsychiatric disorders), following an overnight period of abstinence, to identify regions of the left dorsal lateral prefrontal cortex associated with the anticipation of drug-related rewards (cigarette puff). Next, we recorded the reward positivity-an electrophysiological signal believed to index sensitivity of the ACC to rewards-while participants engaged in two feedback tasks to gain either monetary or cigarette rewards. Lastly, guided by functional magnetic resonance imaging data, a robotic arm positioned a repetitive transcranial magnetic stimulation coil over a subject-specific dorsal lateral prefrontal cortex target, and 50 repetitive transcranial magnetic stimulation pulses were delivered at 10 Hz (excitatory stimulation) immediately before each block of 10 trials of the money condition and at 1 Hz (inhibitory stimulation) before each block of 10 trials of the cigarette condition. RESULTS: Our findings show that abstained smokers exhibited a heightened reward positivity to cigarette rewards relative to monetary rewards, and by applying excitatory or inhibitory repetitive transcranial magnetic stimulation to a subject-specific frontal-cingulate reward pathway, this pattern of results was reversed. CONCLUSIONS: By modulating how the brain links value to drug and nondrug rewards, novel brain-based treatments may finally be on the horizon.

It's all about timing: An electrophysiological examination of feedback-based learning with immediate and delayed feedback.

Feedback regarding an individual's action can occur immediately or with a temporal delay. Processing of feedback that varies in its delivery time is proposed to engage different brain mechanisms. fMRI data implicate the striatum in the processing of immediate feedback, and the medial temporal lobe (MTL) in the processing of delayed feedback. The present study offers an electrophysiological examination of feedback processing in the context of timing, by studying the effects of feedback timing on the feedback-related negativity (FRN), a product of the midbrain dopamine system, and elucidating whether the N170 ERP component could capture MTL activation associated with the processing of delayed feedback. Participants completed a word-object paired association learning task; they received feedback 500ms (immediate feedback condition) following a button press during the learning of two sets of 14 items, and at a delay of 6500ms (delayed feedback condition) during the learning of the other two sets. The results indicated that while learning outcomes did not differ under the two timing conditions, Event Related Potential (ERPs) pointed to differential activation of the examined ERP components. FRN amplitude was found to be larger following the immediate feedback condition when compared with the delayed feedback condition, and sensitive to valence and learning only under the immediate feedback condition. Additionally, the amplitude of the N170 was found larger following the delayed feedback condition when compared with the immediate feedback condition. Taken together, the findings of the present study support the contention that the processing of delayed feedback involves a shift away from midbrain dopamine activation to the recruitment of the MTL.

Effects of delaying binge drinking on adolescent brain development: a longitudinal neuroimaging study.

BACKGROUND: Onset of alcohol use by 14 relative to 21 years of age strongly predicts elevated risk for severe alcohol use problems, with 27% versus 4% of individuals exhibiting alcohol dependence within 10 years of onset. What remains unclear is whether this early alcohol use (i) is a marker for later problems, reflected as a pre-existing developmental predisposition, (ii) causes global neural atrophy or (iii) specifically disturbs neuro-maturational processes implicated in addiction, such as executive functions or reward processing. Since our group has demonstrated that a novel intervention program targeting personality traits associated with adolescent alcohol use can prevent the uptake of drinking and binge drinking by 40 to 60%, a crucial question is whether prevention of early onset alcohol misuse will protect adolescent neurodevelopment and which domains of neurodevelopment can be protected. METHODS: A subsample of 120 youth at high risk for substance misuse and 30 low-risk youth will be recruited from the Co-Venture trial (Montreal, Canada) to take part in this 5-year follow-up neuroimaging study. The Co-Venture trial is a community-based cluster-randomised trial evaluating the effectiveness of school-based personality-targeted interventions on substance use and cognitive outcomes involving approximately 3800 Grade 7 youths. Half of the 120 high-risk participants will have received the preventative intervention program. Cognitive tasks and structural and functional neuroimaging scans will be conducted at baseline, and at 24- and 48-month follow-up. Two functional paradigms will be used: the Stop-Signal Task to measure motor inhibitory control and a modified version of the Monetary Incentive Delay Task to evaluate reward processing. DISCUSSION: The expected results should help identify biological vulnerability factors, and quantify the consequences of early alcohol abuse as well as the benefits of early intervention using brain metrics.

Oscillatory profiles of positive, negative and neutral feedback stimuli during adaptive decision making.

The electrophysiological response to positive and negative feedback during reinforcement learning has been well documented over the past two decades, yet, little is known about the neural response to uninformative events that often follow our actions. To address this issue, we recorded the electroencephalograph (EEG) during a time-estimation task using both informative (positive and negative) and uninformative (neutral) feedback. In the time-frequency domain, uninformative feedback elicited significantly less induced beta-gamma activity than informative feedback. This result suggests that beta-gamma activity is particularly sensitive to feedback that can guide behavioral adjustments, consistent with other work. In contrast, neither theta nor delta activity were sensitive to the difference between negative and neutral feedback, though both frequencies discriminated between positive, and non-positive (neutral or negative) feedback. Interestingly, in the time domain, we observed a linear relationship in the amplitude of the feedback-related negativity (neutral>negative>positive), a component of the event-related brain potential thought to index a specific kind of reinforcement learning signal called a reward prediction error. Taken together, these results suggest that the reinforcement learning system treats neutral feedback as a special case, providing valuable information about the electrophysiological measures used to index the cognitive function of frontal midline cortex.

Reward Sensitivity of ACC as an Intermediate Phenotype between DRD4-521T and Substance Misuse.

The development and expression of the midbrain dopamine system is determined in part by genetic factors that vary across individuals such that dopamine-related genes are partly responsible for addiction vulnerability. However, a complete account of how dopamine-related genes predispose individuals to drug addiction remains to be developed. Adopting an intermediate phenotype approach, we investigated whether reward-related electrophysiological activity of ACC-a cortical region said to utilize dopamine reward signals to learn the value of extended, context-specific sequences of goal-directed behaviors-mediates the influence of multiple dopamine-related functional polymorphisms over substance use. We used structural equation modeling to examine whether two related electrophysiological phenomena associated with the control and reinforcement learning functions of ACC-theta power and the reward positivity-mediated the relationship between the degree of substance misuse and genetic polymorphisms that regulate dopamine processing in frontal cortex. Substance use data were collected from 812 undergraduate students. One hundred ninety-six returned on a subsequent day to participate in an electrophysiological experiment and to provide saliva samples for DNA analysis. We found that these electrophysiological signals mediated a relationship between the DRD4-521T dopamine receptor genotype and substance misuse. Our results provide a theoretical framework that bridges the gap between genes and behavior in drug addiction and illustrate how future interventions might be individually tailored for specific genetic and neurocognitive profiles.

Atypical valuation of monetary and cigarette rewards in substance dependent smokers.

OBJECTIVE: Substance dependent (SD) relative to non-dependent (ND) individuals exhibit an attenuated reward positivity, an electrophysiological signal believed to index sensitivity of anterior cingulate cortex (ACC) to rewards. Here we asked whether this altered neural response reflects a specific devaluation of monetary rewards relative to drug-related rewards by ACC. METHODS: We recorded the reward positivity from SD and ND individuals who currently smoke, following an overnight period of abstinence, while they engaged in two feedback tasks. In a money condition the feedback indicated either a monetary reward or no reward, and in a cigarette condition the feedback indicated either a drug-related reward or no reward. RESULTS: Overall, cigarette relative to monetary rewards elicited a larger reward positivity. Further, for the subjects who engaged in the money condition first, the reward positivity was smaller for the SD compared to the ND participants, but for the subjects who engaged in the cigarette condition first, the reward positivity was larger for the SD compared to the ND participants. CONCLUSIONS: Our results suggest that the initial category of feedback "primed" the response of the ACC to the alternative feedback type on subsequent trials, and that SD and ND individuals responded differently to this priming effect. SIGNIFICANCE: We propose that for people who misuse addictive substances, the prospect of obtaining drug-related rewards engages the ACC to exert control over extended behaviors.