A comparison of visual and auditory EEG interfaces for robot multi-stage task control.

Shared autonomy holds promise for assistive robotics, whereby physically-impaired people can direct robots to perform various tasks for them. However, a robot that is capable of many tasks also introduces many choices for the user, such as which object or location should be the target of interaction. In the context of non-invasive brain-computer interfaces for shared autonomy-most commonly electroencephalography-based-the two most common choices are to provide either auditory or visual stimuli to the user-each with their respective pros and cons. Using the oddball paradigm, we designed comparable auditory and visual interfaces to speak/display the choices to the user, and had users complete a multi-stage robotic manipulation task involving location and object selection. Users displayed differing competencies-and preferences-for the different interfaces, highlighting the importance of considering modalities outside of vision when constructing human-robot interfaces.

Association of bidirectional network cores in the brain with conscious perception and cognition.

The brain comprises a complex network of interacting regions. To understand the roles and mechanisms of this complex network, its structural features related to specific cognitive functions need to be elucidated. Among such relationships, recent developments in neuroscience highlight the link between network bidirectionality and conscious perception. Given the essential roles of both feedforward and feedback signals in conscious perception, it is surmised that subnetworks with bidirectional interactions are critical. However, the link between such subnetworks and conscious perception remains unclear due to the network's complexity. In this study, we propose a framework for extracting subnetworks with strong bidirectional interactions-termed the "cores" of a network-from brain activity. We applied this framework to resting-state and task-based fMRI data to identify regions forming strongly bidirectional cores. We then explored the association of these cores with conscious perception and cognitive functions. The central cores predominantly included cerebral cortical regions, which are crucial for conscious perception, rather than subcortical regions. Furthermore, the cores were composed of previously reported regions in which electrical stimulation altered conscious perception. These results suggest a link between the bidirectional cores and conscious perception. A meta-analysis and comparison of the core structure with a cortical functional connectivity gradient suggested that the central cores were related to lower-order sensorimotor functions. An ablation study emphasized the importance of incorporating bidirectionality, not merely interaction strength for these outcomes. The proposed framework provides novel insight into the roles of network cores with strong bidirectional interactions in conscious perception and lower-order sensorimotor functions.

Integrated information theory (IIT) 4.0: Formulating the properties of phenomenal existence in physical terms.

This paper presents Integrated Information Theory (IIT) 4.0. IIT aims to account for the properties of experience in physical (operational) terms. It identifies the essential properties of experience (axioms), infers the necessary and sufficient properties that its substrate must satisfy (postulates), and expresses them in mathematical terms. In principle, the postulates can be applied to any system of units in a state to determine whether it is conscious, to what degree, and in what way. IIT offers a parsimonious explanation of empirical evidence, makes testable predictions concerning both the presence and the quality of experience, and permits inferences and extrapolations. IIT 4.0 incorporates several developments of the past ten years, including a more accurate formulation of the axioms as postulates and mathematical expressions, the introduction of a unique measure of intrinsic information that is consistent with the postulates, and an explicit assessment of causal relations. By fully unfolding a system's irreducible cause-effect power, the distinctions and relations specified by a substrate can account for the quality of experience.

System Integrated Information.

Integrated information theory (IIT) starts from consciousness itself and identifies a set of properties (axioms) that are true of every conceivable experience. The axioms are translated into a set of postulates about the substrate of consciousness (called a complex), which are then used to formulate a mathematical framework for assessing both the quality and quantity of experience. The explanatory identity proposed by IIT is that an experience is identical to the cause-effect structure unfolded from a maximally irreducible substrate (a Φ-structure). In this work we introduce a definition for the integrated information of a system (φs) that is based on the existence, intrinsicality, information, and integration postulates of IIT. We explore how notions of determinism, degeneracy, and fault lines in the connectivity impact system-integrated information. We then demonstrate how the proposed measure identifies complexes as systems, the φs of which is greater than the φs of any overlapping candidate systems.

Optimal Control Costs of Brain State Transitions in Linear Stochastic Systems.

The brain is a system that performs numerous functions by controlling its states. Quantifying the cost of this control is essential as it reveals how the brain can be controlled based on the minimization of the control cost, and which brain regions are most important to the optimal control of transitions. Despite its great potential, the current control paradigm in neuroscience uses a deterministic framework and is therefore unable to consider stochasticity, severely limiting its application to neural data. Here, to resolve this limitation, we propose a novel framework for the evaluation of control costs based on a linear stochastic model. Following our previous work, we quantified the optimal control cost as the minimal Kullback-Leibler divergence between the uncontrolled and controlled processes. In the linear model, we established an analytical expression for minimal cost and showed that we can decompose it into the cost for controlling the mean and covariance of brain activity. To evaluate the utility of our novel framework, we examined the significant brain regions in the optimal control of transitions from the resting state to seven cognitive task states in human whole-brain imaging data of either sex. We found that, in realizing the different transitions, the lower visual areas commonly played a significant role in controlling the means, while the posterior cingulate cortex commonly played a significant role in controlling the covariances.SIGNIFICANCE STATEMENT The brain performs many cognitive functions by controlling its states. Quantifying the cost of this control is essential as it reveals how the brain can be optimally controlled in terms of the cost, and which brain regions are most important to the optimal control of transitions. Here, we built a novel framework to quantify control cost that takes account of stochasticity of neural activity, which is ignored in previous studies. We established the analytical expression of the stochastic control cost, which enables us to compute the cost in high-dimensional neural data. We identified the significant brain regions for the optimal control in cognitive tasks in human whole-brain imaging data.

Quantifying brain state transition cost via Schrödinger Bridge.

Quantifying brain state transition cost is a fundamental problem in systems neuroscience. Previous studies utilized network control theory to measure the cost by considering a neural system as a deterministic dynamical system. However, this approach does not capture the stochasticity of neural systems, which is important for accurately quantifying brain state transition cost. Here, we propose a novel framework based on optimal control in stochastic systems. In our framework, we quantify the transition cost as the Kullback-Leibler divergence from an uncontrolled transition path to the optimally controlled path, which is known as Schrödinger Bridge. To test its utility, we applied this framework to functional magnetic resonance imaging data from the Human Connectome Project and computed the brain state transition cost in cognitive tasks. We demonstrate correspondence between brain state transition cost and the difficulty of tasks. The results suggest that our framework provides a general theoretical tool for investigating cognitive functions from the viewpoint of transition cost.

Frequency-specific task modulation of human brain functional networks: A fast fMRI study.

How coherent neural oscillations are involved in task execution is a fundamental question in neuroscience. Although several electrophysiological studies have tackled this issue, the brain-wide task modulation of neural coherence remains uncharacterized. Here, with a fast fMRI technique, we studied shifts of brain-wide neural coherence across different task states in the ultraslow frequency range (0.01-0.7 Hz). First, we examined whether the shifts of the brain-wide neural coherence occur in a frequency-dependent manner. We quantified the shift of a region's average neural coherence by the inter-state variance of the mean coherence between the region and the rest of the brain. A clustering analysis based on the variance's spatial correlation between frequency components revealed four frequency bands (0.01-0.15 Hz, 0.15-0.37 Hz, 0.37-0.53 Hz, and 0.53-0.7 Hz) showing band-specific shifts of the brain-wide neural coherence. Next, we investigated the similarity of the inter-state variance's spectra between all pairs of regions. We found that regions showing similar spectra correspond to those forming functional modules of the brain network. Then, we investigated the relationship between identified frequency bands and modules' inter-state variances. We found that modules showing the highest variance are those made up of parieto-occipital regions at 0.01-0.15 Hz, while it is replaced with another consisting of frontal regions above 0.15 Hz. Furthermore, these modules showed specific shifting patterns of the mean coherence across states at 0.01-0.15 Hz and above 0.15 Hz, suggesting that identified frequency bands differentially contribute to neural interactions during task execution. Our results highlight that usage of the fast fMRI enables brain-wide investigation of neural coherence up to 0.7 Hz, which opens a promising track for assessment of the large-scale neural interactions in the ultraslow frequency range.

A general spectral decomposition of causal influences applied to integrated information.

BACKGROUND: Quantifying interactions among many neurons is fundamental to understanding system-level phenomena such as attention, learning and even conscious experience. Causal influences in the brain, quantified as integrated information, are thought to support subjective conscious experience. Recent empirical work has shown that the spectral decomposition of causal influences, for example using Granger causality, can reveal frequency-specific influences that are not observed in the time domain. However, a spectral decomposition of integrated information has not been put forward, limiting its adoption for analyzing neural data. NEW METHOD: We present a general and flexible framework for deriving the spectral decomposition of causal influences in autoregressive processes. RESULTS: We use the framework to derive a spectral decomposition of integrated information. We show that other well-known measures, including Granger causality, can be derived using the same framework. Using simulations, we demonstrate a complex interplay between the spectral decomposition of integrated information and other measures that is not observed in the time domain. COMPARISON WITH EXISTING METHODS: This paper provides a spectral decomposition of integrated information for the first time. Although a spectral decomposition of Granger causality has been derived, that approach is only applicable to uni-directional causal influences, not multi-directional causal influences as required for integrated information. CONCLUSIONS: Our novel framework can be used to derive the spectral decomposition of uni- and multi-directional measures of causal influences. We use this framework to derive a spectral decomposition of integrated information, paving the way for better understanding how frequency-specific causal influences in the brain relate to cognition.

Regulation of cortical activity and arousal by the matrix cells of the ventromedial thalamic nucleus.

The "non-specific" ventromedial thalamic nucleus (VM) has long been considered a candidate for mediating cortical arousal due to its diffuse, superficial projections, but direct evidence was lacking. Here, we show in mice that the activity of VM calbindin1-positive matrix cells is high in wake and REM sleep and low in NREM sleep, and increases before cortical activity at the sleep-to-wake transition. Optogenetic stimulation of VM cells rapidly awoke all mice from NREM sleep and consistently caused EEG activation during slow wave anesthesia, while arousal did not occur from REM sleep. Conversely, chemogenetic inhibition of VM decreased wake duration. Optogenetic activation of the "specific" ventral posteromedial nucleus (VPM) did not cause arousal from either NREM or REM sleep. Thus, matrix cells in VM produce arousal and broad cortical activation during NREM sleep and slow wave anesthesia in a way that accounts for the effects classically attributed to "non-specific" thalamic nuclei.

Differentiation Analysis of Continuous Electroencephalographic Activity Triggered by Video Clip Contents.

While viewing a video clip, we experience a wide variety of contents, from low-level features of the images to high-level ideas such as the storyline. Each change in our experience must be supported by some corresponding change in neurophysiological activity. Differentiation analysis, which quantifies the differences in brain activity by measuring the distances between observed brain states, was applied here to continuous high-density electroencephalographic data recorded while participants watched short video clips. These clips were manipulated in various ways to change the degree of meaningfulness of their contents. We found that neurophysiological differentiation mirrored that of phenomenal differentiation, being higher for meaningful clips and lower for phase-scrambled versions or random noise. The distinction between meaningful and meaningless clips was present even at the individual level, and moreover, differentiation values correlated with individual subjective reports of meaningfulness. Spatial and spectral breakdowns of the overall effect showed frontal and posterior ROIs and highlighted specific roles for different spectral bands. Comparing the results with a multivariate decoding approach reveals that the two methods are capturing different aspects of brain activity and highlights a crucial theoretical distinction between the level and pattern of activity. In future applications, differentiation analysis may be used to evaluate the subjective meaningfulness of stimuli when behavioral responses may be inadequate, as with disorders of consciousness.

Functional split brain in a driving/listening paradigm.

We often engage in two concurrent but unrelated activities, such as driving on a quiet road while listening to the radio. When we do so, does our brain split into functionally distinct entities? To address this question, we imaged brain activity with fMRI in experienced drivers engaged in a driving simulator while listening either to global positioning system instructions (integrated task) or to a radio show (split task). We found that, compared with the integrated task, the split task was characterized by reduced multivariate functional connectivity between the driving and listening networks. Furthermore, the integrated information content of the two networks, predicting their joint dynamics above and beyond their independent dynamics, was high in the integrated task and zero in the split task. Finally, individual subjects' ability to switch between high and low information integration predicted their driving performance across integrated and split tasks. This study raises the possibility that under certain conditions of daily life, a single brain may support two independent functional streams, a "functional split brain" similar to what is observed in patients with an anatomical split.

Mapping Multiplex Hubs in Human Functional Brain Networks.

Typical brain networks consist of many peripheral regions and a few highly central ones, i.e., hubs, playing key functional roles in cerebral inter-regional interactions. Studies have shown that networks, obtained from the analysis of specific frequency components of brain activity, present peculiar architectures with unique profiles of region centrality. However, the identification of hubs in networks built from different frequency bands simultaneously is still a challenging problem, remaining largely unexplored. Here we identify each frequency component with one layer of a multiplex network and face this challenge by exploiting the recent advances in the analysis of multiplex topologies. First, we show that each frequency band carries unique topological information, fundamental to accurately model brain functional networks. We then demonstrate that hubs in the multiplex network, in general different from those ones obtained after discarding or aggregating the measured signals as usual, provide a more accurate map of brain's most important functional regions, allowing to distinguish between healthy and schizophrenic populations better than conventional network approaches.

Stimulus set meaningfulness and neurophysiological differentiation: a functional magnetic resonance imaging study.

A meaningful set of stimuli, such as a sequence of frames from a movie, triggers a set of different experiences. By contrast, a meaningless set of stimuli, such as a sequence of 'TV noise' frames, triggers always the same experience--of seeing 'TV noise'--even though the stimuli themselves are as different from each other as the movie frames. We reasoned that the differentiation of cortical responses underlying the subject's experiences, as measured by Lempel-Ziv complexity (incompressibility) of functional MRI images, should reflect the overall meaningfulness of a set of stimuli for the subject, rather than differences among the stimuli. We tested this hypothesis by quantifying the differentiation of brain activity patterns in response to a movie sequence, to the same movie scrambled in time, and to 'TV noise', where the pixels from each movie frame were scrambled in space. While overall cortical activation was strong and widespread in all conditions, the differentiation (Lempel-Ziv complexity) of brain activation patterns was correlated with the meaningfulness of the stimulus set, being highest in the movie condition, intermediate in the scrambled movie condition, and minimal for 'TV noise'. Stimulus set meaningfulness was also associated with higher information integration among cortical regions. These results suggest that the differentiation of neural responses can be used to assess the meaningfulness of a given set of stimuli for a given subject, without the need to identify the features and categories that are relevant to the subject, nor the precise location of selective neural responses.

Frequency-specific network topologies in the resting human brain.

A community is a set of nodes with dense inter-connections, while there are sparse connections between different communities. A hub is a highly connected node with high centrality. It has been shown that both "communities" and "hubs" exist simultaneously in the brain's functional connectivity network (FCN), as estimated by correlations among low-frequency spontaneous fluctuations in functional magnetic resonance imaging (fMRI) signal changes (0.01-0.10 Hz). This indicates that the brain has a spatial organization that promotes both segregation and integration of information. Here, we demonstrate that frequency-specific network topologies that characterize segregation and integration also exist within this frequency range. In investigating the coherence spectrum among 87 brain regions, we found that two frequency bands, 0.01-0.03 Hz (very low frequency [VLF] band) and 0.07-0.09 Hz (low frequency [LF] band), mainly contributed to functional connectivity. Comparing graph theoretical indices for the VLF and LF bands revealed that the network in the former had a higher capacity for information segregation between identified communities than the latter. Hubs in the VLF band were mainly located within the anterior cingulate cortices, whereas those in the LF band were located in the posterior cingulate cortices and thalamus. Thus, depending on the timescale of brain activity, at least two distinct network topologies contributed to information segregation and integration. This suggests that the brain intrinsically has timescale-dependent functional organizations.

A NIRS-fMRI study of resting state network.

Resting state functional connectivity, which is defined as temporal correlation of spontaneous activity between diverse brain regions, has been reported to form resting state networks (RSNs), consisting of a specific set of brain regions, based on functional magnetic resonance imaging (fMRI). Recently, studies using near-infrared spectroscopy (NIRS) reported that NIRS signals also show temporal correlation between different brain regions. The local relationship between NIRS and fMRI signals has been examined by simultaneously recording these signals when participants perform tasks or respond to stimuli. However, the NIRS-fMRI signal relationship during the resting state has been reported only between NIRS signals obtained within limited regions and whole brain fMRI signals. Therefore, it remains unclear whether NIRS signals obtained at diverse regions correlate with regional fMRI signals close to the NIRS measurement channels, especially in relation to the RSNs. In this study, we tested whether the signals measured by these different modalities during the resting state have the consistent characteristics of the RSNs. Specifically, NIRS signals during the resting state were acquired over the frontal, temporal, and occipital cortices while whole brain fMRI data was simultaneously recorded. First, by projecting the NIRS channel positions over the cerebral cortical surface, we identified the most likely anatomical locations of all NIRS channels used in the study. Next, to investigate the regional signal relationship between NIRS and fMRI, we calculated the cross-correlation between NIRS signals and fMRI signals in the brain regions adjacent to each NIRS channel. For each NIRS channel, we observed the local maxima of correlation coefficients between NIRS and fMRI signals within a radius of 2 voxels from the projection point. Furthermore, we also found that highly correlated voxels with the NIRS signal were mainly localized within brain tissues for all NIRS channels, with the exception of 2 frontal channels. Finally, by calculating the correlation between NIRS signals at a channel and whole brain fMRI signals, we observed that NIRS signals correlate with fMRI signals not only within brain regions adjacent to NIRS channels but also within distant brain regions constituting RSNs, such as the dorsal attention, fronto-parietal control, and default mode networks. These results support the idea that NIRS signals obtained at several cortical regions during the resting state mainly reflect regional spontaneous hemodynamic fluctuations that originate from spontaneous cortical activity, and include information that characterizes the RSNs. Because NIRS is relatively easy to use and a less physically demanding neuroimaging technique, our findings should facilitate a broad application of this technique to examine RSNs, especially for clinical populations and conditions unsuitable for fMRI.

Frequency-specific functional connectivity in the brain during resting state revealed by NIRS.

Analyses of spontaneous hemodynamic fluctuations observed on functional magnetic resonance imaging (fMRI) have revealed the existence of temporal correlations in signal changes between widely separated brain regions during the resting state, termed "resting state functional connectivity." Recent studies have demonstrated that these correlations are also present in the hemodynamic signals measured by near infrared spectroscopy (NIRS). However, it is still uncertain whether frequency-specific characteristics exist in these signals. In the present study, we used multichannel NIRS to investigate the frequency dependency of functional connectivity between diverse regions in the cerebral cortex by decomposing fluctuations of oxygenated hemoglobin (oxy-Hb) and deoxygenated hemoglobin (deoxy-Hb) signals into various frequency bands. First, within a wide frequency range (0.009-0.1Hz), we observed that both oxy-Hb and deoxy-Hb signals showed functional connectivity within local regions and between contralateral hemispheric regions of the cortex. Next, by decomposing measured fluctuations into narrower frequency components, we determined that only oxy-Hb signals showed frequency-specific functional connectivity between the frontal and occipital regions, emerging in a narrow frequency range (0.04-0.1Hz). To clarify the coherency of functional connectivity, we calculated the average coherence values between selected channel pairs. This approach demonstrated that functional connectivity based on the oxy-Hb signals between homologous cortical regions of the contralateral hemisphere (homologous connectivity) showed high coherence over a wide frequency range (0.009-0.1Hz), whereas connectivity between the prefrontal and occipital regions (fronto-posterior connectivity) showed high coherence only within a specific narrow frequency range (0.04-0.1Hz). Our findings suggest that homologous connectivity may reflect synchronization of neural activation over a wide frequency range through direct neuroanatomical connections, whereas fronto-posterior connectivity as revealed by high coherence only within a specific narrow frequency range corresponding to the time scale of typical hemodynamic response to a single event may reflect synchronization of transient neural activation among distant cortical regions. The present study demonstrated that NIRS provides a powerful tool to elucidate network properties of the cortex during resting state.