Theta oscillations shift towards optimal frequency for cognitive control.

Cognitive control allows to flexibly guide behaviour in a complex and ever-changing environment. It is supported by theta band (4-7 Hz) neural oscillations that coordinate distant neural populations. However, little is known about the precise neural mechanisms permitting such flexible control. Most research has focused on theta amplitude, showing that it increases when control is needed, but a second essential aspect of theta oscillations, their peak frequency, has mostly been overlooked. Here, using computational modelling and behavioural and electrophysiological recordings, in three independent datasets, we show that theta oscillations adaptively shift towards optimal frequency depending on task demands. We provide evidence that theta frequency balances reliable set-up of task representation and gating of task-relevant sensory and motor information and that this frequency shift predicts behavioural performance. Our study presents a mechanism supporting flexible control and calls for a reevaluation of the mechanistic role of theta oscillations in adaptive behaviour.

Reward prediction errors drive declarative learning irrespective of agency.

Recent years have witnessed a steady increase in the number of studies investigating the role of reward prediction errors (RPEs) in declarative learning. Specifically, in several experimental paradigms, RPEs drive declarative learning, with larger and more positive RPEs enhancing declarative learning. However, it is unknown whether this RPE must derive from the participant's own response, or whether instead, any RPE is sufficient to obtain the learning effect. To test this, we generated RPEs in the same experimental paradigm where we combined an agency and a nonagency condition. We observed no interaction between RPE and agency, suggesting that any RPE (irrespective of its source) can drive declarative learning. This result holds implications for declarative learning theory.

Signed Reward Prediction Errors in the Ventral Striatum Drive Episodic Memory.

Recent behavioral evidence implicates reward prediction errors (RPEs) as a key factor in the acquisition of episodic memory. Yet, important neural predictions related to the role of RPEs in episodic memory acquisition remain to be tested. Humans (both sexes) performed a novel variable-choice task where we experimentally manipulated RPEs and found support for key neural predictions with fMRI. Our results show that in line with previous behavioral observations, episodic memory accuracy increases with the magnitude of signed (i.e., better/worse-than-expected) RPEs (SRPEs). Neurally, we observe that SRPEs are encoded in the ventral striatum (VS). Crucially, we demonstrate through mediation analysis that activation in the VS mediates the experimental manipulation of SRPEs on episodic memory accuracy. In particular, SRPE-based responses in the VS (during learning) predict the strength of subsequent episodic memory (during recollection). Furthermore, functional connectivity between task-relevant processing areas (i.e., face-selective areas) and hippocampus and ventral striatum increased as a function of RPE value (during learning), suggesting a central role of these areas in episodic memory formation. Our results consolidate reinforcement learning theory and striatal RPEs as key factors subtending the formation of episodic memory.SIGNIFICANCE STATEMENT Recent behavioral research has shown that reward prediction errors (RPEs), a key concept of reinforcement learning theory, are crucial to the formation of episodic memories. In this study, we reveal the neural underpinnings of this process. Using fMRI, we show that signed RPEs (SRPEs) are encoded in the ventral striatum (VS), and crucially, that SRPE VS activity is responsible for the subsequent recollection accuracy of one-shot learned episodic memory associations.

Learning to Synchronize: Midfrontal Theta Dynamics during Rule Switching.

In recent years, several hierarchical extensions of well-known learning algorithms have been proposed. For example, when stimulus-action mappings vary across time or context, the brain may learn two or more stimulus-action mappings in separate modules, and additionally (at a hierarchically higher level) learn to appropriately switch between those modules. However, how the brain mechanistically coordinates neural communication to implement such hierarchical learning remains unknown. Therefore, the current study tests a recent computational model that proposed how midfrontal theta oscillations implement such hierarchical learning via the principle of binding by synchrony (Sync model). More specifically, the Sync model uses bursts at theta frequency to flexibly bind appropriate task modules by synchrony. The 64-channel EEG signal was recorded while 27 human subjects (female: 21, male: 6) performed a probabilistic reversal learning task. In line with the Sync model, postfeedback theta power showed a linear relationship with negative prediction errors, but not with positive prediction errors. This relationship was especially pronounced for subjects with better behavioral fit (measured via Akaike information criterion) of the Sync model. Also consistent with Sync model simulations, theta phase-coupling between midfrontal electrodes and temporoparietal electrodes was stronger after negative feedback. Our data suggest that the brain uses theta power and synchronization for flexibly switching between task rule modules, as is useful, for example, when multiple stimulus-action mappings must be retained and used.SIGNIFICANCE STATEMENT Everyday life requires flexibility in switching between several rules. A key question in understanding this ability is how the brain mechanistically coordinates such switches. The current study tests a recent computational framework (Sync model) that proposed how midfrontal theta oscillations coordinate activity in hierarchically lower task-related areas. In line with predictions of this Sync model, midfrontal theta power was stronger when rule switches were most likely (strong negative prediction error), especially in subjects who obtained a better model fit. Additionally, also theta phase connectivity between midfrontal and task-related areas was increased after negative feedback. Thus, the data provided support for the hypothesis that the brain uses theta power and synchronization for flexibly switching between rules.

Failure to modulate reward prediction errors in declarative learning with theta (6 Hz) frequency transcranial alternating current stimulation.

Recent evidence suggests that reward prediction errors (RPEs) play an important role in declarative learning, but its neurophysiological mechanism remains unclear. Here, we tested the hypothesis that RPEs modulate declarative learning via theta-frequency oscillations, which have been related to memory encoding in prior work. For that purpose, we examined the interaction between RPE and transcranial Alternating Current Stimulation (tACS) in declarative learning. Using a between-subject (real versus sham stimulation group), single-blind stimulation design, 76 participants learned 60 Dutch-Swahili word pairs, while theta-frequency (6 Hz) tACS was administered over the medial frontal cortex (MFC). Previous studies have implicated MFC in memory encoding. We replicated our previous finding of signed RPEs (SRPEs) boosting declarative learning; with larger and more positive RPEs enhancing memory performance. However, tACS failed to modulate the SRPE effect in declarative learning and did not affect memory performance. Bayesian statistics supported evidence for an absence of effect. Our study confirms a role of RPE in declarative learning, but also calls for standardized procedures in transcranial electrical stimulation.

Reward Prediction Error and Declarative Memory.

Learning based on reward prediction error (RPE) was originally proposed in the context of nondeclarative memory. We postulate that RPE may support declarative memory as well. Indeed, recent years have witnessed a number of independent empirical studies reporting effects of RPE on declarative memory. We provide a brief overview of these studies, identify emerging patterns, and discuss open issues such as the role of signed versus unsigned RPEs in declarative learning.

Preparing for hard times: Scalp and intracranial physiological signatures of proactive cognitive control.

Based on reward and difficulty information, people can strategically adjust proactive cognitive control. fMRI research shows that motivated proactive control is implemented through fronto-parietal control networks that are triggered by reward and difficulty cues. Here, we investigate electrophysiological signatures of proactive control. Previously, the contingent negative variation (CNV) in the ERPs and oscillatory power in the theta (4-8 Hz) and alpha band (8-14 Hz) have been suggested as signatures of control implementation. However, experimental designs did not always separate control implementation from motor preparation. Critically, we used a mental calculation task to investigate effects of proactive control implementation on the CNV and on theta and alpha power, in absence of motor preparation. In the period leading up to task onset, we found a more negative CNV, increased theta power, and decreased alpha power for hard versus easy calculations, showing increased proactive control implementation when a difficult task was expected. These three measures also correlated with behavioral performance, both across trials and across subjects. In addition to scalp EEG in healthy participants, we collected intracranial local field potential recordings in an epilepsy patient. We observed a slow-drift component that was more pronounced for hard trials in a hippocampal location, possibly reflecting task-specific preparation for hard mental calculations. The current study thus shows that difficulty information triggers proactive control in absence of motor preparation and elucidates its neurophysiological signatures.

Oscillatory signatures of reward prediction errors in declarative learning.

Reward prediction errors (RPEs) are crucial to learning. Whereas these mismatches between reward expectation and reward outcome are known to drive procedural learning, their role in declarative learning remains underexplored. Earlier work from our lab addressed this, and consistently found that signed reward prediction errors (SRPEs; "better-than-expected" signals) boost declarative learning. In the current EEG study, we sought to explore the neural signatures of SRPEs. Participants studied 60 Dutch-Swahili word pairs while RPE magnitudes were parametrically manipulated. Behaviorally, we replicated our previous findings that SRPEs drive declarative learning, with increased recognition for word pairs accompanied by large, positive RPEs. In the EEG data, at the start of reward feedback processing, we found an oscillatory (theta) signature consistent with unsigned reward prediction errors (URPEs; "different-than-expected" signals). Slightly later during reward feedback processing, we observed oscillatory (high-beta and high-alpha) signatures for SRPEs during reward feedback, similar to SRPE signatures during procedural learning. These findings illuminate the time course of neural oscillations in processing reward during declarative learning, providing important constraints for future theoretical work.

The sound of beauty: How complexity determines aesthetic preference.

Stimulus complexity is an important determinant of aesthetic preference. An influential idea is that increases in stimulus complexity lead to increased preference up to an optimal point after which preference decreases (inverted-U pattern). However, whereas some studies indeed observed this pattern, most studies instead showed an increased preference for more complexity. One complicating issue is that it remains unclear how to define complexity. To address this, we approached complexity and its relation to aesthetic preference from a predictive coding perspective. Here, low- and high-complexity stimuli would correspond to low and high levels of prediction errors, respectively. We expected participants to prefer stimuli which are neither too easy to predict (low prediction error), nor too difficult (high prediction error). To test this, we presented two sequences of tones on each trial that varied in predictability from highly regular (low prediction error) to completely random (high prediction error), and participants had to indicate which of the two sequences they preferred in a two-interval forced-choice task. The complexity of each tone sequence (amount of prediction error) was estimated using entropy. Results showed that participants tended to choose stimuli with intermediate complexity over those of high or low complexity. This confirms the century-old idea that stimulus complexity has an inverted-U relationship to aesthetic preference.

Signed reward prediction errors drive declarative learning.

Reward prediction errors (RPEs) are thought to drive learning. This has been established in procedural learning (e.g., classical and operant conditioning). However, empirical evidence on whether RPEs drive declarative learning-a quintessentially human form of learning-remains surprisingly absent. We therefore coupled RPEs to the acquisition of Dutch-Swahili word pairs in a declarative learning paradigm. Signed RPEs (SRPEs; "better-than-expected" signals) during declarative learning improved recognition in a follow-up test, with increasingly positive RPEs leading to better recognition. In addition, classic declarative memory mechanisms such as time-on-task failed to explain recognition performance. The beneficial effect of SRPEs on recognition was subsequently affirmed in a replication study with visual stimuli.

Occipital alpha power reveals fast attentional inhibition of incongruent distractors.

Recent associative models of cognitive control hypothesize that cognitive control can be learned (optimized) for task-specific settings via associations between perceptual, motor, and control representations, and, once learned, control can be implemented rapidly. Midfrontal brain areas signal the need for control, and control is subsequently implemented by biasing sensory representations, boosting or suppressing activity in brain areas processing task-relevant or task-irrelevant information. To assess the timescale of this process, we employed EEG. In order to pinpoint control implementation in specific sensory areas, we used a flanker task with incongruent flankers shown in only one hemifield (congruent flankers in the other hemifield) isolating their processing in the contralateral hemisphere. ERPs revealed fast modulations specifically in visual processing areas contralateral to the incongruent flankers. To test whether these modulations reflect increased or decreased processing of incongruent flankers, we investigated alpha power, a marker for attentional inhibition. Importantly, we show increased alpha power over visual areas processing incongruent flankers from 300 to 500 ms poststimulus onset. This suggests fast cognitive control by attentional inhibition for information disrupting goal-oriented actions. Additionally, we show that midfrontal theta earlier in the trial is also modulated by incongruency, and that theta power predicts subsequent alpha power modulations. This supports the hypothesis that midfrontal incongruency detection leads to control implementation, and reveals that these mechanisms take place on a fast, within-trial timescale.

Spontaneous eyeblinks during breaking continuous flash suppression are associated with increased detection times.

An eyeblink has a clear effect on low-level information processing because it temporarily occludes all visual information. Recent evidence suggests that eyeblinks can also modulate higher level processes (e.g., attentional resources), and vice versa. Despite these putative effects on different levels of information processing, eyeblinks are typically neglected in vision and in consciousness research. The main aim of this study was to investigate the timing and the effect of eyeblinks in an increasingly popular paradigm in consciousness research, namely breaking continuous flash suppression (b-CFS). Results show that participants generally refrain from blinking during a trial, that is, when they need to detect a suppressed stimulus. However, when they do blink during a trial, we observed a sharp increase in suppression time. This suggests that one needs to control for blinking when comparing detection times between conditions that could elicit phasic changes in blinking.

Predictive information speeds up visual awareness in an individuation task by modulating threshold setting, not processing efficiency.

Theories on visual awareness claim that predicted stimuli reach awareness faster than unpredicted ones. In the current study, we disentangle whether prior information about the upcoming stimulus affects visual awareness of stimulus location (i.e., individuation) by modulating processing efficiency or threshold setting. Analogous research on stimulus identification revealed that prior information modulates threshold setting. However, as identification and individuation are two functionally and neurally distinct processes, the mechanisms underlying identification cannot simply be extrapolated directly to individuation. The goal of this study was therefore to investigate how individuation is influenced by prior information about the upcoming stimulus. To do so, a drift diffusion model was fitted to estimate the processing efficiency and threshold setting for predicted versus unpredicted stimuli in a cued individuation paradigm. Participants were asked to locate a picture, following a cue that was congruent, incongruent or neutral with respect to the picture's identity. Pictures were individuated faster in the congruent and neutral condition compared to the incongruent condition. In the diffusion model analysis, the processing efficiency was not significantly different across conditions. However, the threshold setting was significantly higher following an incongruent cue compared to both congruent and neutral cues. Our results indicate that predictive information about the upcoming stimulus influences visual awareness by shifting the threshold for individuation rather than by enhancing processing efficiency.

The time course of cognitive control implementation.

Optimally recruiting cognitive control is a key factor in efficient task performance. In line with influential cognitive control theories, earlier work assumed that control is relatively slow. We challenge this notion and test whether control also can be implemented more rapidly by investigating the time course of cognitive control. In two experiments, a visual discrimination paradigm was applied. A reward cue was presented with variable intervals to target onset. The results showed that reward cues can rapidly improve performance. Importantly, the reward manipulation was orthogonal to the response, ensuring that the reward effect was due to fast cognitive control implementation rather than to automatic activation of rewarded S-R associations. We also empirically specify the temporal limits of cognitive control, because the reward cue had no effect when it was presented shortly after target onset, during task execution.

Different effects of executive and visuospatial working memory on visual consciousness.

Consciousness and working memory are two widely studied cognitive phenomena. Although they have been closely tied on a theoretical and neural level, empirical work that investigates their relation is largely lacking. In this study, the relationship between visual consciousness and different working memory components is investigated by using a dual-task paradigm. More specifically, while participants were performing a visual detection task to measure their visual awareness threshold, they had to concurrently perform either an executive or visuospatial working memory task. We hypothesized that visual consciousness would be hindered depending on the type and the size of the load in working memory. Results showed that maintaining visuospatial content in working memory hinders visual awareness, irrespective of the amount of information maintained. By contrast, the detection threshold was progressively affected under increasing executive load. Interestingly, increasing executive load had a generic effect on detection speed, calling into question whether its obstructing effect is specific to the visual awareness threshold. Together, these results indicate that visual consciousness depends differently on executive and visuospatial working memory.

Opposite effects of working memory on subjective visibility and priming.

Cognitive theories on consciousness propose a strong link between consciousness and working memory (WM). This link is also present at the neural level: Both consciousness and WM have been implicated in a prefrontal parietal network. However, the link remains empirically unexplored. The present study investigates the relation between consciousness and WM by studying the impact of WM load on one aspect of consciousness, the threshold for subjective visibility. At the same time, we investigated how WM affects cognitive control, a function that has been implicated to consciousness. Results showed an increase in the threshold of subjective visibility when WM load increased. Furthermore, the impact of a prime stimulus on the response to the target was also modulated by WM load. We propose that the observed interference is caused by the functional and neural overlap of these functions.