Phasic Stimulation of Dopaminergic Neurons of the Lateral Substantia Nigra Increases Open Field Exploratory Behaviour and Reduces Habituation Over Time.

Transient nigrostriatal dopaminergic signalling is well known for its role in reinforcement learning and increasingly so for its role in the initiation of voluntary movement. However, how transient bursts of dopamine modulate voluntary movement remains unclear, likely due to the heterogeneity of the nigrostriatal system, the focus of optogenetic studies on locomotion at sub-sec time intervals, and the overlapping roles of phasic dopamine in behaviour and novelty signalling. In this study we investigated how phasic activity in the lateral substantia nigra pars compacta (lateral SNc) over time affects voluntary behaviours during exploration. Using a transgenic mouse model of both sexes expressing channelrhodopsin (ChR2) in dopamine transporter-expressing cells, we stimulated the lateral SNc while mice explored an open field over two consecutive days. We found that phasic activation of the lateral SNc induced an increase in exploratory behaviours including horizontal movement activity, locomotion initiation, and rearing specifically on the first open field exposure, but not on the second day. In addition, stimulated animals did not habituate to the same extent as their ChR2-negative counterparts, as indicated by a lack of decrease in baseline activity. These findings suggest that rather than prompting voluntary movement in general, phasic nigrostriatal dopamine prompts context-appropriate behaviours. In addition, dopamine signalling that modulates movement acts over longer timescales than the transient signal, affecting behaviour even after the signal has ended.

Stabilizing expectations when shifting from analytical to intuitive reasoning: The role of prediction errors in reasoning.

As humans, we rely on intuitive reasoning for most of our decisions. However, when there is a novel or atypical decision to be made, we must rely on a slower and more deliberative thought process-analytical reasoning. As we gain experience with these novel or atypical decisions, our reasoning shifts from analytical to intuitive, which parallels a reduction in the need for cognitive control. Here, we sought to confirm this claim by employing electroencephalographic (EEG) measures of cognitive control as participants performed a simple perceptual decision-making task. Specifically, we had participants categorize "blobs" into families based on their visual attributes so we could examine how their reasoning changed with learning. In a key manipulation, halfway through the experiment we introduced novel blob families to categorize, thus temporarily increasing the need for analytical reasoning (i.e., cognitive control). Congruent with past research, we focused our EEG analyses on frontal theta activity as it has been linked to cognitive control and analytical thinking. As hypothesized, we found a transition from analytical to intuitive decision-making systems with learning as indexed by a decrease in frontal theta power. Further, when the novel blobs were introduced at the midpoint of the experiment, we found that decisions about these stimuli recruited analytical reasoning as indicated by increased theta power in comparison to decisions about well-practiced stimuli. We propose our findings to reflect prediction errors to decision demands-a monitoring process that determines whether our expectations of demands are met. Shifting from analytical to intuitive reasoning thus reflects the stabilization of our expectations of decision demands, which can be violated with unexpected demands when encountering novel stimuli.

Using EEG to decode semantics during an artificial language learning task.

BACKGROUND: As we learn a new nonnative language (L2), we begin to build a new map of concepts onto orthographic representations. Eventually, L2 can conjure as rich a semantic representation as our native language (L1). However, the neural processes for mapping a new orthographic representation to a familiar meaning are not well understood or characterized. METHODS: Using electroencephalography and an artificial language that maps symbols to English words, we show that it is possible to use machine learning models to detect a newly formed semantic mapping as it is acquired. RESULTS: Through a trial-by-trial analysis, we show that we can detect when a new semantic mapping is formed. Our results show that, like word meaning representations evoked by a L1, the localization of the newly formed neural representations is highly distributed, but the representation may emerge more slowly after the onset of the symbol. Furthermore, our mapping of word meanings to symbols removes the confound of the semantics to the visual characteristics of the stimulus, a confound that has been difficult to disentangle previously. CONCLUSION: We have shown that the L1 semantic representation conjured by a newly acquired L2 word can be detected using decoding techniques, and we give the first characterization of the emergence of that mapping. Our work opens up new possibilities for the study of semantic representations during L2 learning.

What happens when right means wrong? The impact of conflict arising from competing feedback responses.

Humans often rely on feedback to learn. Indeed, in learning the difference between feedback and an expected outcome is computed to inform future actions. Further, recent work has found that reward and feedback have a unique role in modulating conflict processing and cognitive control. However, it is still not clear how conflict, especially concerning the processing and evaluation of feedback, impacts learning. To address this, we examined the effects of feedback competition on feedback evaluation in a reinforcement learning task. Specifically, we had participants play a simple two-choice gambling game while electroencephalographic (EEG) data were recorded. On half of the experiment blocks, we reversed the meaning of performance feedback for each trial from its prepotent meaning to induce response conflict akin to the Stroop effect (e.g., '✓' meant incorrect). Behaviourally, we found that participants' accuracy was reduced as a result of incongruent feedback. Paralleling this, an analysis of our EEG revealed that incongruent feedback resulted in a reduction in amplitude of the reward positivity and the P300, components of the human event-related brain potential implicated in reward processing. Our results demonstrate the negative impact of conflict on feedback evaluation and the impact of this on subsequent performance.

Dissociated neural signals of conflict and surprise in effortful decision Making: Theta activity reflects surprise while alpha and beta activity reflect conflict.

What makes a decision difficult? Two key factors are conflict and surprise: conflict emerges with multiple competing responses and surprise occurs with unexpected events. Conflict and surprise, however, are often thought of as parsimonious accounts of decision making rather than an integrated narrative. We sought to determine whether conflict and/or surprise concurrently or independently elicit effortful decision making. Participants made a series of diagnostic decisions from physiological readings while electroencephalographic (EEG) data were recorded. To induce conflict and surprise, we manipulated task difficulty by varying the distance between a presented physiological reading and the category border that separated the two diagnoses. Whereas frontal theta oscillations reflected surprise - when presented readings were far from the expected mean, parietal alpha and beta oscillations indicated conflict - when readings were near the category border. Our findings provide neural evidence that both conflict and surprise engage cognitive control to employ effort in decision making.

The ERP, frequency, and time-frequency correlates of feedback processing: Insights from a large sample study.

Human learning, at least in part, appears to be dependent on the evaluation of how outcomes of our actions align with our expectations. Over the past 23 years, electroencephalography (EEG) has been used to probe the neural signatures of feedback processing. Seminal work demonstrated a difference in the human event-related potential (ERP) dependent on whether people were processing correct or incorrect feedback. Since then, these feedback evoked ERPs have been associated with reinforcement learning and conflict monitoring, tied to subsequent behavioral adaptations, and shown to be sensitive to a wide range of factors (e.g., Parkinson's disease). Recently, research has turned to frequency decomposition techniques to examine how changes in the EEG power spectra are related to underlying learning mechanisms. Although the literature on the neural correlates of feedback processing is vast, there are still methodological discrepancies and differences in results across studies. Here, we provide reference results and an investigation of methodological considerations for the ERP (reward positivity) and frequency (delta and theta power) correlates of feedback evaluation with a large sample size. Specifically, participants (n = 500) performed a two-armed bandit task while we recorded EEG. Our findings provide key information about the data characteristics and relationships that exist between the neural signatures of feedback evaluation. Additionally, we conclude with selected methodological recommendations for standardization of future research. All data and scripts are freely provided to facilitate open science.

The impact of wellness on neural learning systems.

Over the past 20 years there has been an increasing push for people to achieve or maintain "wellness" - a state in which one has not only physical but also mental and social well-being. While it may seem obvious that maintaining a state of wellness is beneficial, little research has been done to probe how maintaining a state of wellness impacts our brain. Here, we specifically examined the impact of wellness on a neural system within the medial-frontal cortex responsible for human reinforcement learning. Sixty-two undergraduate students completed the Perceived Wellness Survey after which they completed a computer-based learnable gambling game while electroencephalographic data were recorded. Within the game, participants were presented with a series of choices that either led to financial gains or losses. An analysis of our behavioral data indicated that participants were able to learn the underlying structure of the gambling game given that we observed improvements in performance. Concurrent with this, we observed an electroencephalographic response evoked by the evaluation of gambling outcomes - the reward positivity. Importantly, we found significant relationships between several aspects of wellness and the amplitude of the reward positivity. Given that the reward positivity is thought to reflect the function of a reinforcement learning system within the medial-frontal cortex, our results suggest that wellness impacts neural function - in this instance one of the systems responsible for human learning.

Electroencephalography correlates of transcranial direct-current stimulation enhanced surgical skill learning: A replication and extension study.

Transcranial direct-current stimulation (tDCS), an increasingly applied form of non-invasive brain stimulation, can augment the acquisition of motor skills. Motor learning investigations of tDCS are limited to simple skills, where mechanisms are increasingly understood. Investigations of meaningful, complex motor skills possessed by humans, such as surgical skills, are limited. This replication and extension of our previous findings used electroencephalography (EEG) to determine how tDCS and complex surgical training alters electrical activity in the sensorimotor network to enhance complex surgical skill acquisition. In twenty-two participants, EEG was recorded during baseline performance of simulation-based laparoscopic surgical skills. Participants were randomized to receive 20 min of primary motor cortex targeting anodal tDCS or sham concurrent to 1 h of surgical skill training. EEG was reassessed following training, during a post-training repetition of the surgical tasks. Our results replicated our previous study suggesting that compared to sham, anodal tDCS enhanced the acquisition of unimanual surgical skill. Surgical training modulated delta frequency band activity in sensorimotor regions. Next, the performance of unimanual and bimanual skills evoked unique EEG profiles, primarily within the beta frequency-band in parietal regions. Finally, tDCS-paired surgical training independently modulated delta and alpha frequency-bands in sensorimotor regions. Application of tDCS during surgical skill training is feasible, safe and tolerable. In conclusion, we are the first to explore electrical brain activity during performance of surgical skills, how electrical activity may change during surgical training and how tDCS alters the brain to enhance skill acquisition. The results provide preliminary evidence of neural markers that can be targeted by neuromodulation to optimize complex surgical training.

Passively learned spatial navigation cues evoke reinforcement learning reward signals.

Since the suggestion by Tolman (1948) that both rodents and humans create cognitive maps during navigation, the specifics of how navigators learn about their environment has been mired in debate. One facet of this debate is whether or not the creation of cognitive maps - also known as allocentric navigation - involves reinforcement learning. Here, we demonstrate a role for reinforcement learning during allocentric navigation using event-related brain potentials (ERPs). In the present experiment, participants navigated in a virtual environment that allowed the use of three different navigation strategies (allocentric, egocentric-response, & egocentric-cue), in which their goal was to locate and remember a hidden platform. Following the navigation phase of the experiment, participants were shown "cue images" representative of the three navigation strategies. Specifically, we examined whether or not these passively learned strategy images elicited a reward positivity - an ERP component associated with reinforcement learning and the anterior cingulate cortex. We found that when allocentric navigators were shown previously learned cues predicting the goal location a reward positivity was elicited. The present findings demonstrate that allocentric navigational cues carry long-term value after navigation and lend support to the claim that reinforcement learning plays a role in the acquisition of allocentric navigation and thus the generation of cognitive maps.

Thinking theta and alpha: Mechanisms of intuitive and analytical reasoning.

Humans have a unique ability to engage in different modes of thinking. Intuitive thinking (coined System 1, see Kahneman, 2011) is fast, automatic, and effortless whereas analytical thinking (coined System 2) is slow, contemplative, and effortful. We extend seminal pupillometry research examining these modes of thinking by using electroencephalography (EEG) to decipher their respective underlying neural mechanisms. We demonstrate that System 1 thinking is characterized by an increase in parietal alpha EEG power reflecting autonomic access to long-term memory and a release of attentional resources whereas System 2 thinking is characterized by an increase in frontal theta EEG power indicative of the engagement of cognitive control and working memory processes. Consider our results in terms of an example - a child may need cognitive control and working memory when contemplating a mathematics problem yet an adult can drive a car with little to no attention by drawing on easily accessed memories. Importantly, the unravelling of intuitive and analytical thinking mechanisms and their neural signatures will provide insight as to how different modes of thinking drive our everyday lives.

Chasing the zone: Reduced beta power predicts baseball batting performance.

Mental state prior to sports skill execution is related to subsequent performance. For example, relationships between pre-performance electroencephalogram (EEG) power and subsequent movement outcomes in golf putting, pistol shooting, and basketball free throw shooting have been previously reported. With that said, the existing body of research examining the pre-performance EEG - performance relationship has been focused on the execution of internally as opposed to externally-paced motor skills. Given that the execution of internally and externally-paced movements are dependent on different neural pathways, in the present study we examined whether or not pre-performance EEG power predicted ensuing performance of an externally-paced motor skill - baseball batting. Sixty-seven baseball players had EEG data recorded for 120 s prior to batting practice. Performance was assessed by three expert coaches and the accuracy of coach performance ratings was verified via Generalizability Theory. An analysis of our data revealed an inverse relationship between frontal EEG power in the beta range and subsequent batting performance - reduced beta power was associated with better batting performance whereas increased beta power was associated with worse batting performance. Our results are in line with prior research that has demonstrated a relationship between increased EEG power in the beta range and the subsequent commitment of motor errors in addition to the aforementioned work examining pre-performance EEG and the execution of internally-paced motor skills.

Alcohol hangover impacts learning and reward processing within the medial-frontal cortex.

It is common knowledge that alcohol intoxication impairs motor coordination, judgment, and decision making. Indeed, an abundance of literature links intoxication to impaired cognitive control that leads to accidents and injury. A broadening body of research, however, suggests that the impact of alcohol may continue beyond the point of intoxication and into the period of alcohol hangover. Here, we examined differences in the amplitude of reward positivity-a component of the human ERP associated with learning-between control and hangover participants. During performance of a learnable gambling task, we found a reduction in the reward positivity during alcohol hangover. Additionally, participants experiencing alcohol hangover demonstrated reduced performance in the experimental task in comparison to their nonhangover counterparts. Our results suggest that the neural systems that underlie performance monitoring and reward-based learning are impaired during alcohol hangover.

The application of reward learning in the real world: Changes in the reward positivity amplitude reflect learning in a medical education context.

Evidence ranging from behavioural adaptations to neurocognitive theories has made significant advances into our understanding of feedback-based learning. For instance, over the past twenty years research using electroencephalography has demonstrated that the amplitude of a component of the human event-related brain potential - the reward positivity - appears to change with learning in a manner predicted by reinforcement learning theory (Holroyd and Coles, 2002; Sutton and Barto, 1998). However, while the reward positivity (also known as the feedback related negativity) is well studied, whether the component reflects an underlying learning process or whether it is simply sensitive to feedback evaluation is still unclear. Here, we sought to provide support that the reward positivity is reflective of an underlying learning process and further we hoped to demonstrate this in a real-world medical education context. In the present study, students with no medical training viewed a series of patient cards that contained ten physiological readings relevant for diagnosing liver and biliary disease types, selected the most appropriate diagnostic classification, and received feedback as to whether their decisions were correct or incorrect. Our behavioural results revealed that our participants were able to learn to diagnose liver and biliary disease types. Importantly, we found that the amplitude of the reward positivity diminished in a concomitant manner with the aforementioned behavioural improvements. In sum, our data support theoretical predictions (e.g., Holroyd and Coles, 2002), suggest that the reward positivity is an index of a neural learning system, and further validate that this same system is involved in learning across a wide range of contexts.

When theory and biology differ: The relationship between reward prediction errors and expectancy.

Comparisons between expectations and outcomes are critical for learning. Termed prediction errors, the violations of expectancy that occur when outcomes differ from expectations are used to modify value and shape behaviour. In the present study, we examined how a wide range of expectancy violations impacted neural signals associated with feedback processing. Participants performed a time estimation task in which they had to guess the duration of one second while their electroencephalogram was recorded. In a key manipulation, we varied task difficulty across the experiment to create a range of different feedback expectancies - reward feedback was either very expected, expected, 50/50, unexpected, or very unexpected. As predicted, the amplitude of the reward positivity, a component of the human event-related brain potential associated with feedback processing, scaled inversely with expectancy (e.g., unexpected feedback yielded a larger reward positivity than expected feedback). Interestingly, the scaling of the reward positivity to outcome expectancy was not linear as would be predicted by some theoretical models. Specifically, we found that the amplitude of the reward positivity was about equivalent for very expected and expected feedback, and for very unexpected and unexpected feedback. As such, our results demonstrate a sigmoidal relationship between reward expectancy and the amplitude of the reward positivity, with interesting implications for theories of reinforcement learning.

Choosing MUSE: Validation of a Low-Cost, Portable EEG System for ERP Research.

In recent years there has been an increase in the number of portable low-cost electroencephalographic (EEG) systems available to researchers. However, to date the validation of the use of low-cost EEG systems has focused on continuous recording of EEG data and/or the replication of large system EEG setups reliant on event-markers to afford examination of event-related brain potentials (ERP). Here, we demonstrate that it is possible to conduct ERP research without being reliant on event markers using a portable MUSE EEG system and a single computer. Specifically, we report the results of two experiments using data collected with the MUSE EEG system-one using the well-known visual oddball paradigm and the other using a standard reward-learning task. Our results demonstrate that we could observe and quantify the N200 and P300 ERP components in the visual oddball task and the reward positivity (the mirror opposite component to the feedback-related negativity) in the reward-learning task. Specifically, single sample t-tests of component existence (all p's < 0.05), computation of Bayesian credible intervals, and 95% confidence intervals all statistically verified the existence of the N200, P300, and reward positivity in all analyses. We provide with this research paper an open source website with all the instructions, methods, and software to replicate our findings and to provide researchers with an easy way to use the MUSE EEG system for ERP research. Importantly, our work highlights that with a single computer and a portable EEG system such as the MUSE one can conduct ERP research with ease thus greatly extending the possible use of the ERP methodology to a variety of novel contexts.

The scarcity heuristic impacts reward processing within the medial-frontal cortex.

Objects that are rare are often perceived to be inherently more valuable than objects that are abundant - a bias brought about in part by the scarcity heuristic. In the present study, we sought to test whether perception of rarity impacted reward evaluation within the human medial-frontal cortex. Here, participants played a gambling game in which they flipped rare and abundant 'cards' on a computer screen to win financial rewards while electroencephalographic data were recorded. Unbeknownst to participants, reward outcome and frequency was random and equivalent for both rare and abundant cards; thus, only a perception of scarcity was true. Analysis of the electroencephalographic data indicated that the P300 component of the event-related brain potential differed in amplitude for wins and losses following the selection of rare cards, but not following the selection of abundant cards. Importantly, then, we found that the perception of card rarity impacted reward processing even though reward feedback was independent of and subsequent to card selection. Our data indicate a top-down influence of the scarcity heuristic on reward evaluation, and specifically the processing of reward magnitude, within the human medial-frontal cortex.