Design and evaluation of a global workspace agent embodied in a realistic multimodal environment.

As the apparent intelligence of artificial neural networks (ANNs) advances, they are increasingly likened to the functional networks and information processing capabilities of the human brain. Such comparisons have typically focused on particular modalities, such as vision or language. The next frontier is to use the latest advances in ANNs to design and investigate scalable models of higher-level cognitive processes, such as conscious information access, which have historically lacked concrete and specific hypotheses for scientific evaluation. In this work, we propose and then empirically assess an embodied agent with a structure based on global workspace theory (GWT) as specified in the recently proposed "indicator properties" of consciousness. In contrast to prior works on GWT which utilized single modalities, our agent is trained to navigate 3D environments based on realistic audiovisual inputs. We find that the global workspace architecture performs better and more robustly at smaller working memory sizes, as compared to a standard recurrent architecture. Beyond performance, we perform a series of analyses on the learned representations of our architecture and share findings that point to task complexity and regularization being essential for feature learning and the development of meaningful attentional patterns within the workspace.

Toward a universal theory of consciousness.

While falsifiability has been broadly discussed as a desirable property of a theory of consciousness, in this paper, we introduce the meta-theoretic concept of "Universality" as an additional desirable property for a theory of consciousness. The concept of universality, often assumed in physics, posits that the fundamental laws of nature are consistent and apply equally everywhere in the universe and remain constant over time. This assumption is crucial in science, acting as a guiding principle for developing and testing theories. When applied to theories of consciousness, universality can be defined as the ability of a theory to determine whether any fully described dynamical system is conscious or non-conscious. Importantly, for a theory to be universal, the determinant of consciousness needs to be defined as an intrinsic property of a system as opposed to replying on the interpretation of the external observer. The importance of universality originates from the consideration that given that consciousness is a natural phenomenon, it could in principle manifest in any physical system that satisfies a certain set of conditions whether it is biological or non-biological. To date, apart from a few exceptions, most existing theories do not possess this property. Instead, they tend to make predictions as to the neural correlates of consciousness based on the interpretations of brain functions, which makes those theories only applicable to brain-centric systems. While current functionalist theories of consciousness tend to be heavily reliant on our interpretations of brain functions, we argue that functionalist theories could be converted to a universal theory by specifying mathematical formulations of the constituent concepts. While neurobiological and functionalist theories retain their utility in practice, we will eventually need a universal theory to fully explain why certain types of systems possess consciousness.

Deep CANALs: a deep learning approach to refining the canalization theory of psychopathology.

Psychedelic therapy has seen a resurgence of interest in the last decade, with promising clinical outcomes for the treatment of a variety of psychopathologies. In response to this success, several theoretical models have been proposed to account for the positive therapeutic effects of psychedelics. One of the more prominent models is "RElaxed Beliefs Under pSychedelics," which proposes that psychedelics act therapeutically by relaxing the strength of maladaptive high-level beliefs encoded in the brain. The more recent "CANAL" model of psychopathology builds on the explanatory framework of RElaxed Beliefs Under pSychedelics by proposing that canalization (the development of overly rigid belief landscapes) may be a primary factor in psychopathology. Here, we make use of learning theory in deep neural networks to develop a series of refinements to the original CANAL model. Our primary theoretical contribution is to disambiguate two separate optimization landscapes underlying belief representation in the brain and describe the unique pathologies which can arise from the canalization of each. Along each dimension, we identify pathologies of either too much or too little canalization, implying that the construct of canalization does not have a simple linear correlation with the presentation of psychopathology. In this expanded paradigm, we demonstrate the ability to make novel predictions regarding what aspects of psychopathology may be amenable to psychedelic therapy, as well as what forms of psychedelic therapy may ultimately be most beneficial for a given individual.

Hippocampal volume mediates the relationship of parental rejection in childhood with social cognition in healthy adults.

Childhood abuse reduces hippocampal and amygdala volumes and impairs social cognition, including the ability to recognize facial expressions. However, these associations have been studied primarily in individuals with a history of severe abuse and psychiatric symptoms; researchers have not determined whether these associations can also be observed in healthy adults. In the present study, we analyzed data from 400 healthy adults (208 men and 192 women) at Tamagawa University. Parental rejection reflecting childhood abuse was assessed using the short form of Egna Minnen Beträffande Uppfostran, while social cognition was assessed using the "Fake Smile Detection Task." Hippocampal and amygdala volumes were extracted from T1-weighted magnetic resonance imaging data using FreeSurfer. We found that greater parental rejection resulted in smaller hippocampal and amygdala volumes and poorer performance in the Fake Smile Detection Task. Structural equation modeling analysis supported the model that hippocampal volume mediates maternal rejection effect on performance on the Fake Smile Detection Task, with involvement of the amygdala. These findings are in line with the structural and functional connectivity found between the hippocampus and amygdala and their joint involvement in social cognition. Therefore, parental rejection may affect hippocampal and amygdala volumes and social cognitive function even in symptom-free adults.

Effective connectivity in a duration selective cortico-cerebellar network.

How the human brain represents millisecond unit of time is far from clear. A recent neuroimaging study revealed the existence in the human premotor cortex of a topographic representation of time i.e., neuronal units selectively responsive to specific durations and topographically organized on the cortical surface. By using high resolution functional Magnetic Resonance Images here, we go beyond this previous work, showing duration preferences across a wide network of cortical and subcortical brain areas: from cerebellum to primary visual, parietal, premotor and prefrontal cortices. Most importantly, we identify the effective connectivity structure between these different brain areas and their duration selective neural units. The results highlight the role of the cerebellum as the network hub and that of medial premotor cortex as the final stage of duration recognition. Interestingly, when a specific duration is presented, only the communication strength between the units selective to that specific duration and to the neighboring durations is affected. These findings link for the first time, duration preferences within single brain region with connectivity dynamics between regions, suggesting a communication mode that is partially duration specific.

Review of Performance Improvement of a Noninvasive Brain-computer Interface in Communication and Motor Control for Clinical Applications.

Brain-computer interfaces (BCI) enable direct communication between the brain and a computer or other external devices. They can extend a person's degree of freedom by either strengthening or substituting the human peripheral working capacity. Moreover, their potential clinical applications in medical fields include rehabilitation, affective computing, communication, and control. Over the last decade, noninvasive BCI systems such as electroencephalogram (EEG) have progressed from simple statistical models to deep learning models, with performance improvement over time and enhanced computational power. However, numerous challenges pertaining to the clinical use of BCI systems remain, e.g., the lack of sufficient data to learn more possible features for robust and reliable classification. However, compared with fields such as computer vision and speech recognition, the training samples in the medical BCI field are limited as they target patients who face difficulty generating EEG data compared with healthy control. Because deep learning models incorporate several parameters, they require considerably more data than other conventional methods. Thus, deep learning models have not been thoroughly leveraged in medical BCI. This study summarizes the state-of-the-art progress of the BCI system over the last decade, highlighting critical challenges and solutions.

The neuroanatomy of social trust predicts depression vulnerability.

Trust attitude is a social personality trait linked with the estimation of others' trustworthiness. Trusting others, however, can have substantial negative effects on mental health, such as the development of depression. Despite significant progress in understanding the neurobiology of trust, whether the neuroanatomy of trust is linked with depression vulnerability remains unknown. To investigate a link between the neuroanatomy of trust and depression vulnerability, we assessed trust and depressive symptoms and employed neuroimaging to acquire brain structure data of healthy participants. A high depressive symptom score was used as an indicator of depression vulnerability. The neuroanatomical results observed with the healthy sample were validated in a sample of clinically diagnosed depressive patients. We found significantly higher depressive symptoms among low trusters than among high trusters. Neuroanatomically, low trusters and depressive patients showed similar volume reduction in brain regions implicated in social cognition, including the dorsolateral prefrontal cortex (DLPFC), dorsomedial PFC, posterior cingulate, precuneus, and angular gyrus. Furthermore, the reduced volume of the DLPFC and precuneus mediated the relationship between trust and depressive symptoms. These findings contribute to understanding social- and neural-markers of depression vulnerability and may inform the development of social interventions to prevent pathological depression.

The effect of feedback valence and source on perception and metacognition: An fMRI investigation.

Receiving feedback from our environment that informs us about the outcomes of our actions helps us assess our abilities (e.g., metacognition) and to flexibly adapt our behavior, consequently increasing our chances of success. However, a detailed examination of the effect of feedback on the brain activation during perceptual and confidence judgments as well as the interrelations between perceptual accuracy, prospective and retrospective confidence remains unclear. Here we used functional magnetic resonance imaging (fMRI) to examine the neural response to feedback valence and source in visual contrast discrimination together with prospective confidence judgments at the beginning of each block and retrospective confidence judgments after every decision. Positive feedback was associated with higher activation (or lower deactivation depending on the area) in areas previously involved in attention, performance monitoring and visual regions during the perceptual judgment than during the confidence judgment. Changes in prospective confidence were positively related to changes in perceptual accuracy as well as to the corresponding retrospective confidence. Thus, feedback information impacted multiple, qualitatively different brain processing states, and we also revealed the dynamic interplay between prospective, perceptual accuracy and retrospective self-assessment.

Meta-learning, social cognition and consciousness in brains and machines.

The intersection between neuroscience and artificial intelligence (AI) research has created synergistic effects in both fields. While neuroscientific discoveries have inspired the development of AI architectures, new ideas and algorithms from AI research have produced new ways to study brain mechanisms. A well-known example is the case of reinforcement learning (RL), which has stimulated neuroscience research on how animals learn to adjust their behavior to maximize reward. In this review article, we cover recent collaborative work between the two fields in the context of meta-learning and its extension to social cognition and consciousness. Meta-learning refers to the ability to learn how to learn, such as learning to adjust hyperparameters of existing learning algorithms and how to use existing models and knowledge to efficiently solve new tasks. This meta-learning capability is important for making existing AI systems more adaptive and flexible to efficiently solve new tasks. Since this is one of the areas where there is a gap between human performance and current AI systems, successful collaboration should produce new ideas and progress. Starting from the role of RL algorithms in driving neuroscience, we discuss recent developments in deep RL applied to modeling prefrontal cortex functions. Even from a broader perspective, we discuss the similarities and differences between social cognition and meta-learning, and finally conclude with speculations on the potential links between intelligence as endowed by model-based RL and consciousness. For future work we highlight data efficiency, autonomy and intrinsic motivation as key research areas for advancing both fields.

Deep learning and the Global Workspace Theory.

Recent advances in deep learning have allowed artificial intelligence (AI) to reach near human-level performance in many sensory, perceptual, linguistic, and cognitive tasks. There is a growing need, however, for novel, brain-inspired cognitive architectures. The Global Workspace Theory (GWT) refers to a large-scale system integrating and distributing information among networks of specialized modules to create higher-level forms of cognition and awareness. We argue that the time is ripe to consider explicit implementations of this theory using deep-learning techniques. We propose a roadmap based on unsupervised neural translation between multiple latent spaces (neural networks trained for distinct tasks, on distinct sensory inputs and/or modalities) to create a unique, amodal Global Latent Workspace (GLW). Potential functional advantages of GLW are reviewed, along with neuroscientific implications.

A Technical Critique of Some Parts of the Free Energy Principle.

We summarize the original formulation of the free energy principle and highlight some technical issues. We discuss how these issues affect related results involving generalised coordinates and, where appropriate, mention consequences for and reveal, up to now unacknowledged, differences from newer formulations of the free energy principle. In particular, we reveal that various definitions of the "Markov blanket" proposed in different works are not equivalent. We show that crucial steps in the free energy argument, which involve rewriting the equations of motion of systems with Markov blankets, are not generally correct without additional (previously unstated) assumptions. We prove by counterexamples that the original free energy lemma, when taken at face value, is wrong. We show further that this free energy lemma, when it does hold, implies the equality of variational density and ergodic conditional density. The interpretation in terms of Bayesian inference hinges on this point, and we hence conclude that it is not sufficiently justified. Additionally, we highlight that the variational densities presented in newer formulations of the free energy principle and lemma are parametrised by different variables than in older works, leading to a substantially different interpretation of the theory. Note that we only highlight some specific problems in the discussed publications. These problems do not rule out conclusively that the general ideas behind the free energy principle are worth pursuing.

Efficient search for informational cores in complex systems: Application to brain networks.

An important step in understanding the nature of the brain is to identify "cores" in the brain network, where brain areas strongly interact with each other. Cores can be considered as essential sub-networks for brain functions. In the last few decades, an information-theoretic approach to identifying cores has been developed. In this approach, interactions between parts are measured by an information loss function, which quantifies how much information would be lost if interactions between parts were removed. Then, a core called a "complex" is defined as a subsystem wherein the amount of information loss is locally maximal. Although identifying complexes can be a novel and useful approach, its application is practically impossible because computation time grows exponentially with system size. Here we propose a fast and exact algorithm for finding complexes, called Hierarchical Partitioning for Complex search (HPC). HPC hierarchically partitions systems to narrow down candidates for complexes. The computation time of HPC is polynomial, enabling us to find complexes in large systems (up to several hundred) in a practical amount of time. We prove that HPC is exact when an information loss function satisfies a mathematical property, monotonicity. We show that mutual information is one such information loss function. We also show that a broad class of submodular functions can be considered as such information loss functions, indicating the expandability of our framework to the class. We applied HPC to electrocorticogram recordings from a monkey and demonstrated that HPC revealed temporally stable and characteristic complexes.

Information Closure Theory of Consciousness.

Information processing in neural systems can be described and analyzed at multiple spatiotemporal scales. Generally, information at lower levels is more fine-grained but can be coarse-grained at higher levels. However, only information processed at specific scales of coarse-graining appears to be available for conscious awareness. We do not have direct experience of information available at the scale of individual neurons, which is noisy and highly stochastic. Neither do we have experience of more macro-scale interactions, such as interpersonal communications. Neurophysiological evidence suggests that conscious experiences co-vary with information encoded in coarse-grained neural states such as the firing pattern of a population of neurons. In this article, we introduce a new informational theory of consciousness: Information Closure Theory of Consciousness (ICT). We hypothesize that conscious processes are processes which form non-trivial informational closure (NTIC) with respect to the environment at certain coarse-grained scales. This hypothesis implies that conscious experience is confined due to informational closure from conscious processing to other coarse-grained scales. Information Closure Theory of Consciousness (ICT) proposes new quantitative definitions of both conscious content and conscious level. With the parsimonious definitions and a hypothesize, ICT provides explanations and predictions of various phenomena associated with consciousness. The implications of ICT naturally reconcile issues in many existing theories of consciousness and provides explanations for many of our intuitions about consciousness. Most importantly, ICT demonstrates that information can be the common language between consciousness and physical reality.

Temporal variability of brain networks predicts individual differences in bistable perception.

When ambiguous visual stimuli are presented to the eyes, conscious perception can spontaneously alternate across the competing interpretations - which was known as bistable perception. The spontaneous alternation of perception might indicate a connection between bistable perception and the dynamic interaction of brain networks. Here, we hypothesized that individual differences in perceptual dynamics may be reflected in dynamics of spontaneous neural activities. To test this idea, we investigated the relationship between the percept duration and the reconfiguration patterns of dynamic brain networks as measured by the functional connectivity (FC) during the resting state. Firstly, we found that individual difference of percept duration is associated with the temporal variability of the brain regions which were previously reported in studies of bistable perception, including anterior cingulate cortex (ACC), dorsal medial prefrontal cortex (DMPFC), dorsal lateral prefrontal cortex (DLPFC), superior parietal lobule (SPL), inferior parietal lobule (IPL), precuneus, insula, and V5. Secondly, there is a positive relationship between the temporal variability within the frontal-parietal network (FPN) and the percept duration. Thirdly, our results indicated that individual difference of bistable perception was related to the dynamic interaction between large-scale functional networks including default mode network (DMN), FPN, cingulo-opercular network (CON), dorsal attention network (DAN), salience network (SN), memory retrieval network (MRN). Altogether, our results demonstrated that inter-individual variability in bistable perception was associated with dynamic coupling of brain regions and networks involved in primary visual processing, spatial attention, and cognitive control.

An Information Theoretic Approach to Reveal the Formation of Shared Representations.

Modality-invariant categorical representations, i.e., shared representation, is thought to play a key role in learning to categorize multi-modal information. We have investigated how a bimodal autoencoder can form a shared representation in an unsupervised manner with multi-modal data. We explored whether altering the depth of the network and mixing the multi-modal inputs at the input layer affect the development of the shared representations. Based on the activation of units in the hidden layers, we classified them into four different types: visual cells, auditory cells, inconsistent visual and auditory cells, and consistent visual and auditory cells. Our results show that the number and quality of the last type (i.e., shared representation) significantly differ depending on the depth of the network and are enhanced when the network receives mixed inputs as opposed to separate inputs for each modality, as occurs in typical two-stage frameworks. In the present work, we present a way to utilize information theory to understand the abstract representations formed in the hidden layers of the network. We believe that such an information theoretic approach could potentially provide insights into the development of more efficient and cost-effective ways to train neural networks using qualitative measures of the representations that cannot be captured by analyzing only the final outputs of the networks.

Information generation as a functional basis of consciousness.

What is the biological advantage of having consciousness? Functions of consciousness have been elusive due to the subjective nature of consciousness and ample empirical evidence showing the presence of many nonconscious cognitive performances in the human brain. Drawing upon empirical literature, here, we propose that a core function of consciousness be the ability to internally generate representations of events possibly detached from the current sensory input. Such representations are constructed by generative models learned through sensory-motor interactions with the environment. We argue that the ability to generate information underlies a variety of cognitive functions associated with consciousness such as intention, imagination, planning, short-term memory, attention, curiosity, and creativity, all of which contribute to non-reflexive behavior. According to this view, consciousness emerged in evolution when organisms gained the ability to perform internal simulations using internal models, which endowed them with flexible intelligent behavior. To illustrate the notion of information generation, we take variational autoencoders (VAEs) as an analogy and show that information generation corresponds the decoding (or decompression) part of VAEs. In biological brains, we propose that information generation corresponds to top-down predictions in the predictive coding framework. This is compatible with empirical observations that recurrent feedback activations are linked with consciousness whereas feedforward processing alone seems to occur without evoking conscious experience. Taken together, the information generation hypothesis captures many aspects of existing ideas about potential functions of consciousness and provides new perspectives on the functional roles of consciousness.

Chronotopic maps in human supplementary motor area.

Time is a fundamental dimension of everyday experiences. We can unmistakably sense its passage and adjust our behavior accordingly. Despite its ubiquity, the neuronal mechanisms underlying the capacity to perceive time remains unclear. Here, in two experiments using ultrahigh-field 7-Tesla (7T) functional magnetic resonance imaging (fMRI), we show that in the medial premotor cortex (supplementary motor area [SMA]) of the human brain, neural units tuned to different durations are orderly mapped in contiguous portions of the cortical surface so as to form chronomaps. The response of each portion in a chronomap is enhanced by neighboring durations and suppressed by nonpreferred durations represented in distant portions of the map. These findings suggest duration-sensitive tuning as a possible neural mechanism underlying the recognition of time and demonstrate, for the first time, that the representation of an abstract feature such as time can be instantiated by a topographical arrangement of duration-sensitive neural populations.

Representations of time in human frontoparietal cortex.

Precise time estimation is crucial in perception, action and social interaction. Previous neuroimaging studies in humans indicate that perceptual timing tasks involve multiple brain regions; however, whether the representation of time is localized or distributed in the brain remains elusive. Using ultra-high-field functional magnetic resonance imaging combined with multivariate pattern analyses, we show that duration information is decoded in multiple brain areas, including the bilateral parietal cortex, right inferior frontal gyrus and, albeit less clearly, the medial frontal cortex. Individual differences in the duration judgment accuracy were positively correlated with the decoding accuracy of duration in the right parietal cortex, suggesting that individuals with a better timing performance represent duration information in a more distinctive manner. Our study demonstrates that although time representation is widely distributed across frontoparietal regions, neural populations in the right parietal cortex play a crucial role in time estimation.