Model Virtues in Computational Cognitive Neuroscience.

There is an abundance of computational models in cognitive neuroscience. A framework for what is desirable in a model, what justifies the introduction of a new one, or what makes one better than another is lacking, however. In this article, we examine key qualities ("virtues") that are desirable in computational models, and how these are interrelated. To keep the scope of the article manageable, we focus on the field of cognitive control, where we identified six "model virtues": empirical accuracy, empirical scope, functional analysis, causal detail, biological plausibility, and psychological plausibility. We first illustrate their use in published work on Stroop modeling and then discuss what expert modelers in the field of cognitive control said about them in a series of qualitative interviews. We found that virtues are interrelated and that their value depends on the modeler's goals, in ways that are not typically acknowledged in the literature. We recommend that researchers make the reasons for their modeling choices more explicit in published work. Our work is meant as a first step. Although our focus here is on cognitive control, we hope that our findings will spark discussion of virtues in other fields as well.

Do you feel up when you go up? A pilot study of a virtual reality manic-like mood induction paradigm.

OBJECTIVES: In order to understand the working mechanisms of mania, it is necessary to perform studies during the onset of manic (-like) mood states. However, clinical mania is difficult to examine experimentally. A viable method to study manic mood like states is mood induction, but mood induction tasks thus far show variable effectiveness. METHODS: In this pilot study, a new paradigm to induce mood through virtual reality (VR) is examined. Both state characteristics, namely changes in emotion, and trait characteristics, such as high and low scores on the hypomanic personality scale (HPS), were measured in 65 students. These students participated in either a neutral VR mood induction or an activating VR mood induction in which excitement, goal directedness, and tension (being aspects of mania) were induced. All participants performed a risk-taking behavioural task, Balloon Analogue Risk Task (BART). RESULTS: The experimental VR task induced excitement and tension. In participants with higher sensitivity to hypomanic personality (HPS), irritation increased in response to activation whereas it decreased in the low HPS group, and excitement increased more steeply in the low HPS group. There were no effects on the behavioural task. CONCLUSIONS: The VR task is effective in inducing relevant state aspects of hypomania and is suitable as a paradigm for future experimental studies. Activation of dual affective states (excitement and tension) is an essential aspect in manic-like mood induction paradigms.

Short-term transcutaneous vagus nerve stimulation increases pupil size but does not affect EEG alpha power: A replication of Sharon et al. (2021, Journal of Neuroscience).

BACKGROUND: Transcutaneous auricular vagus nerve stimulation (taVNS) has been tested as a potential treatment for pharmaco-resistant epilepsy and depression. Its clinical efficacy is thought to depend on taVNS-induced activation of the locus coeruleus and other neuromodulator systems. However, unlike for invasive VNS in rodents, there is little evidence for an effect of taVNS on noradrenergic activity. OBJECTIVE: We attempted to replicate recently published findings by Sharon et al. (2021), showing that short bursts of taVNS transiently increased pupil size and decreased EEG alpha power, two correlates of central noradrenergic activity. METHODS: Following the original study, we used a single-blind, sham-controlled, randomized cross-over design. Human volunteers (n = 29) received short-term (3.4 s) taVNS at the maximum level below the pain threshold, while we collected resting-state pupil-size and EEG data. To analyze the data, we used scripts provided by Sharon and colleagues. RESULTS: Consistent with Sharon et al. (2021), pupil dilation was significantly larger during taVNS than during sham stimulation (p = .009; Bayes factor supporting the difference = 7.45). However, we failed to replicate the effect of taVNS on EEG alpha power (p = .37); the data were four times more likely under the null hypothesis (BF10 = 0.28). CONCLUSION: Our findings support the effectiveness of short-term taVNS in inducing transient pupil dilation, a correlate of phasic noradrenergic activity. However, we failed to replicate the recent finding by Sharon et al. (2021) that taVNS attenuates EEG alpha activity. Overall, this study highlights the need for continued research on the neural mechanisms underlying taVNS efficacy and its potential as a treatment option for pharmaco-resistant conditions. It also highlights the need for direct replications of influential taVNS studies.

A Critical Period for Robust Curriculum-Based Deep Reinforcement Learning of Sequential Action in a Robot Arm.

Many everyday activities are sequential in nature. That is, they can be seen as a sequence of subactions and sometimes subgoals. In the motor execution of sequential action, context effects are observed in which later subactions modulate the execution of earlier subactions (e.g., reaching for an overturned mug, people will optimize their grasp to achieve a comfortable end state). A trajectory (movement) adaptation of an often-used paradigm in the study of sequential action, the serial response time task, showed several context effects of which centering behavior is of special interest. Centering behavior refers to the tendency (or strategy) of subjects to move their arm or mouse cursor to a position equidistant to all stimuli in the absence of predictive information, thereby reducing movement time to all possible targets. In the current study, we investigated sequential action in a virtual robotic agent trained using proximal policy optimization, a state-of-the-art deep reinforcement learning algorithm. The agent was trained to reach for appearing targets, similar to a serial response time task given to humans. We found that agents were more likely to develop centering behavior similar to human subjects after curricularized learning. In our curriculum, we first rewarded agents for reaching targets before introducing a penalty for energy expenditure. When the penalty was applied with no curriculum, many agents failed to learn the task due to a lack of action space exploration, resulting in high variability of agents' performance. Our findings suggest that in virtual agents, similar to infants, early energetic exploration can promote robust later learning. This may have the same effect as infants' curiosity-based learning by which they shape their own curriculum. However, introducing new goals cannot wait too long, as there may be critical periods in development after which agents (as humans) cannot flexibly learn to incorporate new objectives. These lessons are making their way into machine learning and offer exciting new avenues for studying both human and machine learning of sequential action.

Permutation Entropy as a Universal Disorder Criterion: How Disorders at Different Scale Levels Are Manifestations of the Same Underlying Principle.

What do bacteria, cells, organs, people, and social communities have in common? At first sight, perhaps not much. They involve totally different agents and scale levels of observation. On second thought, however, perhaps they share everything. A growing body of literature suggests that living systems at different scale levels of observation follow the same architectural principles and process information in similar ways. Moreover, such systems appear to respond in similar ways to rising levels of stress, especially when stress levels approach near-lethal levels. To explain such communalities, we argue that all organisms (including humans) can be modeled as hierarchical Bayesian controls systems that are governed by the same biophysical principles. Such systems show generic changes when taxed beyond their ability to correct for environmental disturbances. Without exception, stressed organisms show rising levels of 'disorder' (randomness, unpredictability) in internal message passing and overt behavior. We argue that such changes can be explained by a collapse of allostatic (high-level integrative) control, which normally synchronizes activity of the various components of a living system to produce order. The selective overload and cascading failure of highly connected (hub) nodes flattens hierarchical control, producing maladaptive behavior. Thus, we present a theory according to which organic concepts such as stress, a loss of control, disorder, disease, and death can be operationalized in biophysical terms that apply to all scale levels of organization. Given the presumed universality of this mechanism, 'losing control' appears to involve the same process anywhere, whether involving bacteria succumbing to an antibiotic agent, people suffering from physical or mental disorders, or social systems slipping into warfare. On a practical note, measures of disorder may serve as early warning signs of system failure even when catastrophic failure is still some distance away.

The exploration-exploitation trade-off in a foraging task is affected by mood-related arousal and valence.

The exploration-exploitation trade-off shows conceptual, functional, and neural analogies with the persistence-flexibility trade-off. We investigated whether mood, which is known to modulate the persistence-flexibility balance, would similarly affect the exploration-exploitation trade-off in a foraging task. More specifically, we tested whether interindividual differences in foraging behavior can be predicted by mood-related arousal and valence. In 119 participants, we assessed mood-related interindividual differences in exploration-exploitation using a foraging task that included minimal task constraints to reduce paradigm-induced biases of individual control tendencies. We adopted the marginal value theorem as a model-based analysis approach, which approximates optimal foraging behavior by tackling the patch-leaving problem. To assess influences of mood on foraging, participants underwent either a positive or negative mood induction. Throughout the experiment, we assessed arousal and valence levels as predictors for explorative/exploitative behavior. Our mood manipulation affected participants' arousal and valence ratings as expected. Moreover, mood-related arousal was found to predict exploration while valence predicted exploitation, which only partly matched our expectations and thereby the proposed conceptual overlap with flexibility and persistence, respectively. The current study provides a first insight into how processes related to arousal and valence differentially modulate foraging behavior. Our results imply that the relationship between exploration-exploitation and flexibility-persistence is more complicated than the semantic overlap between these terms might suggest, thereby calling for further research on the functional, neural, and neurochemical underpinnings of both trade-offs.

How higher goals are constructed and collapse under stress: A hierarchical Bayesian control systems perspective.

In this paper, we show that organisms can be modeled as hierarchical Bayesian control systems with small world and information bottleneck (bow-tie) network structure. Such systems combine hierarchical perception with hierarchical goal setting and hierarchical action control. We argue that hierarchical Bayesian control systems produce deep hierarchies of goal states, from which it follows that organisms must have some form of 'highest goals'. For all organisms, these involve internal (self) models, external (social) models and overarching (normative) models. We show that goal hierarchies tend to decompose in a top-down manner under severe and prolonged levels of stress. This produces behavior that favors short-term and self-referential goals over long term, social and/or normative goals. The collapse of goal hierarchies is universally accompanied by an increase in entropy (disorder) in control systems that can serve as an early warning sign for tipping points (disease or death of the organism). In humans, learning goal hierarchies corresponds to personality development (maturation). The failure of goal hierarchies to mature properly corresponds to personality deficits. A top-down collapse of such hierarchies under stress is identified as a common factor in all forms of episodic mental disorders (psychopathology). The paper concludes by discussing ways of testing these hypotheses empirically.

Predictive Movements and Human Reinforcement Learning of Sequential Action.

Sequential action makes up the bulk of human daily activity, and yet much remains unknown about how people learn such actions. In one motor learning paradigm, the serial reaction time (SRT) task, people are taught a consistent sequence of button presses by cueing them with the next target response. However, the SRT task only records keypress response times to a cued target, and thus it cannot reveal the full time-course of motion, including predictive movements. This paper describes a mouse movement trajectory SRT task in which the cursor must be moved to a cued location. We replicated keypress SRT results, but also found that predictive movement-before the next cue appears-increased during the experiment. Moreover, trajectory analyses revealed that people developed a centering strategy under uncertainty. In a second experiment, we made prediction explicit, no longer cueing targets. Thus, participants had to explore the response alternatives and learn via reinforcement, receiving rewards and penalties for correct and incorrect actions, respectively. Participants were not told whether the sequence of stimuli was deterministic, nor if it would repeat, nor how long it was. Given the difficulty of the task, it is unsurprising that some learners performed poorly. However, many learners performed remarkably well, and some acquired the full 10-item sequence within 10 repetitions. Comparing the high- and low-performers' detailed results in this reinforcement learning (RL) task with the first experiment's cued trajectory SRT task, we found similarities between the two tasks, suggesting that the effects in Experiment 1 are due to predictive, rather than reactive processes. Finally, we found that two standard model-free reinforcement learning models fit the high-performing participants, while the four low-performing participants provide better fit with a simple negative recency bias model.

The Impact of Human-Robot Synchronization on Anthropomorphization.

To elucidate the working mechanism behind anthropomorphism, this study investigated whether human participants would anthropomorphize a robot more if they move synchronously versus non-synchronously with it, and whether this is affected by which of the two initiates the movements. We tested two competing hypotheses. The feature-overlap hypothesis predicts that moving in synchrony would increase perceived self-other feature overlap, which in turn might spread activation to codes of features related to humans-which should increase anthropomorphization. In contrast, the autonomy hypothesis predicts that unpredictability increases anthropomorphization, and thus that whenever the robot initiates movements, or when the human initiates movements to which the robot moves non-synchronously, there is an increased perception of the robot as a more human-like, intentionally acting creature, which in turn should increase anthropomorphization. We performed a study with synchrony as within-subjects factor, and initiator (robot or human) as between-subjects factor. To study the impact of synchrony on self-other overlap and perception of human likeness, participants completed two tasks that served as implicit measures of state anthropomorphization, and two questionnaires that served as explicit measures of state anthropomorphization toward the robot. The two implicit measures were the joint Simon task and one-shot Dictator Game. Additionally, participants filled in a trait anthropomorphization questionnaire, to enable correction for baseline tendencies to anthropomorphize. The synchrony manipulation did not affect the joint Simon effect, although there was an effect on average reaction time (RT), where in the group in which the robot initiated the movement, RTs were slower when the human and robot moved non-synchronously. The Dictator Game offer and the state anthropomorphization questionnaires were not affected by the synchrony manipulation. There was, however, a positive correlation between current anthropomorphization of the robot and amount of money offered to it. Given that most measures were not systematically affected by our manipulation, it appears that either our design was suboptimal, or that synchronization does not affect the anthropomorphization of a robot.

Navigating abstract virtual environment: an eeg study.

Perceptions of different environments are different for different people. An abstract designed environment, with a degree of freedom from any visual reference in the physical world requests a completely different perception than a fully or semi-designed environment that has some correlation with the physical world. Maximal evidence on the manner in which the human brain is involved/operates in dealing with such novel perception comes from neuropsychology. Harnessing the tools and techniques involved in the domain of neuropsychology, the paper presents nee evidence on the role of pre-central gyrus in the perception of abstract spatial environments. In order to do so, the research team developed three different categories of designed environment with different characteristics: (1) Abstract environment, (2) Semi-designed environment, (3) Fully designed environment, as experimental sample environments. Perception of Fully-designed and semi-designed environments is almost the same, [maybe] since the brain can find a correlation between designed environments and already experienced physical world. In addition to this, the response to questionnaires accompanied with a list of buzzwords that have been provided after the experiments, also describe the characteristics of the chosen sample environments. Additionally, these results confirm the suitability of continuous electroencephalography (EEG) for studying Perception from the perspective of architectural environments.

Neurotransmitters as food supplements: the effects of GABA on brain and behavior.

Gamma-aminobutyric acid (GABA) is the main inhibitory neurotransmitter in the human cortex. The food supplement version of GABA is widely available online. Although many consumers claim that they experience benefits from the use of these products, it is unclear whether these supplements confer benefits beyond a placebo effect. Currently, the mechanism of action behind these products is unknown. It has long been thought that GABA is unable to cross the blood-brain barrier (BBB), but the studies that have assessed this issue are often contradictory and range widely in their employed methods. Accordingly, future research needs to establish the effects of oral GABA administration on GABA levels in the human brain, for example using magnetic resonance spectroscopy. There is some evidence in favor of a calming effect of GABA food supplements, but most of this evidence was reported by researchers with a potential conflict of interest. We suggest that any veridical effects of GABA food supplements on brain and cognition might be exerted through BBB passage or, more indirectly, via an effect on the enteric nervous system. We conclude that the mechanism of action of GABA food supplements is far from clear, and that further work is needed to establish the behavioral effects of GABA.

A continuous-time neural model for sequential action.

Action selection, planning and execution are continuous processes that evolve over time, responding to perceptual feedback as well as evolving top-down constraints. Existing models of routine sequential action (e.g. coffee- or pancake-making) generally fall into one of two classes: hierarchical models that include hand-built task representations, or heterarchical models that must learn to represent hierarchy via temporal context, but thus far lack goal-orientedness. We present a biologically motivated model of the latter class that, because it is situated in the Leabra neural architecture, affords an opportunity to include both unsupervised and goal-directed learning mechanisms. Moreover, we embed this neurocomputational model in the theoretical framework of the theory of event coding (TEC), which posits that actions and perceptions share a common representation with bidirectional associations between the two. Thus, in this view, not only does perception select actions (along with task context), but actions are also used to generate perceptions (i.e. intended effects). We propose a neural model that implements TEC to carry out sequential action control in hierarchically structured tasks such as coffee-making. Unlike traditional feedforward discrete-time neural network models, which use static percepts to generate static outputs, our biological model accepts continuous-time inputs and likewise generates non-stationary outputs, making short-timescale dynamic predictions.

Everyday robotic action: lessons from human action control.

Robots are increasingly capable of performing everyday human activities such as cooking, cleaning, and doing the laundry. This requires the real-time planning and execution of complex, temporally extended sequential actions under high degrees of uncertainty, which provides many challenges to traditional approaches to robot action control. We argue that important lessons in this respect can be learned from research on human action control. We provide a brief overview of available psychological insights into this issue and focus on four principles that we think could be particularly beneficial for robot control: the integration of symbolic and subsymbolic planning of action sequences, the integration of feedforward and feedback control, the clustering of complex actions into subcomponents, and the contextualization of action-control structures through goal representations.

The impact of alertness on cognitive control.

Many previous studies have found that an increase in phasic or tonic alertness impairs cognitive control, even though overall response times are decreased. This counterintuitive pattern of behavior is still poorly understood. Using a computational model, we show that the behavioral pattern follows directly from two simple and well-supported assumptions: increased alertness reduces the time needed for stimulus encoding; and cognitive control takes time to develop. The simulation results suggest that, although the arousal system and cognitive control system may be anatomically distinct, their effects on information processing may interact to produce a seemingly complicated pattern of behavior.

Consciousness of targets during the attentional blink: a gradual or all-or-none dimension?

Models of consciousness differ in whether they predict a gradual change or a discontinuous transition between nonconscious and conscious perception. Sergent and Dehaene (Psychological Science, 15, 720-728, 2004) asked subjects to rate on a continuous scale the subjective visibility of target words presented during an attentional blink. They found that these words were either detected as well as targets outside the attentional-blink period or not detected at all, and interpreted these results as support for a discontinuous transition between nonconscious and conscious processing. We present results from 4 attentional-blink experiments showing that this all-or-none rating pattern disappears with the use of an alternative measure of consciousness (post-decision wagering) and a more difficult identification task. Instead, under these circumstances, subjects used the consciousness rating scales in a continuous fashion. These results are more consistent with models that assume a gradual change between nonconscious and conscious perception during the attentional blink.