Learning to segment self-generated from externally caused optic flow through sensorimotor mismatch circuits.

Efficient sensory detection requires the capacity to ignore task-irrelevant information, for example when optic flow patterns created by egomotion need to be disentangled from object perception. To investigate how this is achieved in the visual system, predictive coding with sensorimotor mismatch detection is an attractive starting point. Indeed, experimental evidence for sensorimotor mismatch signals in early visual areas exists, but it is not understood how they are integrated into cortical networks that perform input segmentation and categorization. Our model advances a biologically plausible solution by extending predictive coding models with the ability to distinguish self-generated from externally caused optic flow. We first show that a simple three neuron circuit produces experience-dependent sensorimotor mismatch responses, in agreement with calcium imaging data from mice. This microcircuit is then integrated into a neural network with two generative streams. The motor-to-visual stream consists of parallel microcircuits between motor and visual areas and learns to spatially predict optic flow resulting from self-motion. The second stream bidirectionally connects a motion-selective higher visual area (mHVA) to V1, assigning a crucial role to the abundant feedback connections to V1: the maintenance of a generative model of externally caused optic flow. In the model, area mHVA learns to segment moving objects from the background, and facilitates object categorization. Based on shared neurocomputational principles across species, the model also maps onto primate visual cortex. Our work extends Hebbian predictive coding to sensorimotor settings, in which the agent actively moves - and learns to predict the consequences of its own movements.

Dynamical predictive coding with reservoir computing performs noise-robust multi-sensory speech recognition.

Multi-sensory integration is a perceptual process through which the brain synthesizes a unified perception by integrating inputs from multiple sensory modalities. A key issue is understanding how the brain performs multi-sensory integrations using a common neural basis in the cortex. A cortical model based on reservoir computing has been proposed to elucidate the role of recurrent connectivity among cortical neurons in this process. Reservoir computing is well-suited for time series processing, such as speech recognition. This inquiry focuses on extending a reservoir computing-based cortical model to encompass multi-sensory integration within the cortex. This research introduces a dynamical model of multi-sensory speech recognition, leveraging predictive coding combined with reservoir computing. Predictive coding offers a framework for the hierarchical structure of the cortex. The model integrates reliability weighting, derived from the computational theory of multi-sensory integration, to adapt to multi-sensory time series processing. The model addresses a multi-sensory speech recognition task, necessitating the management of complex time series. We observed that the reservoir effectively recognizes speech by extracting time-contextual information and weighting sensory inputs according to sensory noise. These findings indicate that the dynamic properties of recurrent networks are applicable to multi-sensory time series processing, positioning reservoir computing as a suitable model for multi-sensory integration.

Prediction by Young Autistic Children from Visual and Spoken Input.

Recent theoretical accounts suggest that differences in the processing of probabilistic events underlie the core and associated traits of autism spectrum disorder (ASD). These theories hypothesize that autistic individuals are differentially impacted by disruptions in probabilistic input relative to neurotypical peers. According to this view, autistic individuals assign disproportionate weight to prediction errors such that novel input is overweighted relative to the aggregation of prior input; this is referred to as 'hyperplasticity' of learning. Prediction among autistic individuals has primarily been examined in nonverbal, visual contexts with older children and adults. The present study examined 32 autistic and 32 cognitively-matched neurotypical (NT) children's ability to generate predictions and adjust to changes in predictive relationships in auditory stimuli using two eye gaze tasks. In both studies, children were trained and tested on an auditory-visual cue which predicted the location of a reward stimulus. In Experiment 1 the cue was non-linguistic (instrumental sound) whereas in Experiment 2 the cue was linguistically-relevant (speaker gender). In both experiments, the cue-reward contingency was switched after the first block of trials, and predictive behavior was evaluated across a second block of trials. Results: Analyses of children's looking behavior revealed similar performance in both groups on the non-linguistic task (Exp. 1). In the linguistically-relevant task (Exp. 2), predictive looking was less disrupted by the contingency switch for autistic children than NT children. Results suggest that autistic children may demonstrate hyperplastic learning in linguistically-relevant contexts, relative to NT peers.

How Sports Involvement and Brand Fit Influence the Effectiveness of Sports Sponsorship from the Perspective of Predictive Coding Theory: An Event-Related Potential (ERP)-Based Study.

Background/Objectives: With the rapid expansion of the global sports market, the significance of sports sponsorship has attracted growing attention. However, during the golden age of the sports industry's development in China, international sports brand giants such as Nike, Adidas, and Under Armour have rapidly captured a substantial share of the Chinese sports consumer market through their distinctive product designs and varied marketing strategies. This has resulted in a highly competitive environment for China's sports goods industry. Therefore, fostering the improved development of domestic sports brands has become a crucial issue deserving of thorough scholarly investigation. This study examines how consumers' differing levels of sports involvement and the degree of fit between the sponsoring brand and the sponsored event affect their cognitive and emotional responses to sports sponsorships. Methods: By employing Predictive Coding Theory and ERP (event-related potential) brainwave technology, this study delves into the psychological and neurobiological levels to analyze the impact of consumer sports involvement on the processing of sponsorship information. Results: The results indicate significant differences in cognitive and emotional responses between high-involvement and low-involvement consumers. Additionally, the fit between the sponsoring brand and the sponsored event also significantly affects consumers' cognitive and emotional responses. These differences stem from consumers' complex and sophisticated predictive coding models. Conclusions: This study not only provides scientific evidence for sports brands in selecting and executing sponsorship activities, but also offers new perspectives for evaluating and optimizing sponsorship effectiveness.

Confidence and second-order errors in cortical circuits.

Minimization of cortical prediction errors has been considered a key computational goal of the cerebral cortex underlying perception, action, and learning. However, it is still unclear how the cortex should form and use information about uncertainty in this process. Here, we formally derive neural dynamics that minimize prediction errors under the assumption that cortical areas must not only predict the activity in other areas and sensory streams but also jointly project their confidence (inverse expected uncertainty) in their predictions. In the resulting neuronal dynamics, the integration of bottom-up and top-down cortical streams is dynamically modulated based on confidence in accordance with the Bayesian principle. Moreover, the theory predicts the existence of cortical second-order errors, comparing confidence and actual performance. These errors are propagated through the cortical hierarchy alongside classical prediction errors and are used to learn the weights of synapses responsible for formulating confidence. We propose a detailed mapping of the theory to cortical circuitry, discuss entailed functional interpretations, and provide potential directions for experimental work.

Physiological properties of auditory neurons responding to omission deviants in the anesthetized rat.

The detection of novel, low probability events in the environment is critical for survival. To perform this vital task, our brain is continuously building and updating a model of the outside world; an extensively studied phenomenon commonly referred to as predictive coding. Predictive coding posits that the brain is continuously extracting regularities from the environment to generate predictions. These predictions are then used to supress neuronal responses to redundant information, filtering those inputs, which then automatically enhances the remaining, unexpected inputs. We have recently described the ability of auditory neurons to generate predictions about expected sensory inputs by detecting their absence in an oddball paradigm using omitted tones as deviants. Here, we studied the responses of individual neurons to omitted tones by presenting individual sequences of repetitive pure tones, using both random and periodic omissions, presented at both fast and slow rates in the inferior colliculus and auditory cortex neurons of anesthetized rats. Our goal was to determine whether feature-specific dependence of these predictions exists. Results showed that omitted tones could be detected at both high (8 Hz) and slow repetition rates (2 Hz), with detection being more robust at the non-lemniscal auditory pathway.

Language experience shapes predictive coding of rhythmic sound sequences.

Perceptual systems heavily rely on prior knowledge and predictions to make sense of the environment. Predictions can originate from multiple sources of information, including contextual short-term priors, based on isolated temporal situations, and context-independent long-term priors, arising from extended exposure to statistical regularities. While the effects of short-term predictions on auditory perception have been well-documented, how long-term predictions shape early auditory processing is poorly understood. To address this, we recorded magnetoencephalography data from native speakers of two languages with different word orders (Spanish: functor-initial vs Basque: functor-final) listening to simple sequences of binary sounds alternating in duration with occasional omissions. We hypothesized that, together with contextual transition probabilities, the auditory system uses the characteristic prosodic cues (duration) associated with the native language's word order as an internal model to generate long-term predictions about incoming non-linguistic sounds. Consistent with our hypothesis, we found that the amplitude of the mismatch negativity elicited by sound omissions varied orthogonally depending on the speaker's linguistic background and was most pronounced in the left auditory cortex. Importantly, listening to binary sounds alternating in pitch instead of duration did not yield group differences, confirming that the above results were driven by the hypothesized long-term 'duration' prior. These findings show that experience with a given language can shape a fundamental aspect of human perception - the neural processing of rhythmic sounds - and provides direct evidence for a long-term predictive coding system in the auditory cortex that uses auditory schemes learned over a lifetime to process incoming sound sequences.

Examining predictive coding accounts of typical and autistic neurocognitive development.

Predictive coding has emerged as a prominent theoretical framework for understanding perception and its neural underpinnings. There has been a recent surge of interest in the predictive coding framework across the mind sciences. However, comparatively little of the research in this field has investigated the neural underpinnings of predictive coding in young neurotypical and autistic children. This paper provides an overview of predictive coding accounts of typical and autistic neurocognitive development and includes a review of the current electrophysiological evidence supporting these accounts. Based on the current evidence, it is clear that more research in pediatrics is needed to evaluate predictive coding accounts of neurocognitive development fully. If supported, these accounts could have wide-ranging practical implications for pedagogy, parenting, artificial intelligence, and clinical approaches to helping autistic children manage the barrage of everyday sensory information.

The Many Roles of Precision in Action.

Active inference describes (Bayes-optimal) behaviour as being motivated by the minimisation of surprise of one's sensory observations, through the optimisation of a generative model (of the hidden causes of one's sensory data) in the brain. One of active inference's key appeals is its conceptualisation of precision as biasing neuronal communication and, thus, inference within generative models. The importance of precision in perceptual inference is evident-many studies have demonstrated the importance of ensuring precision estimates are correct for normal (healthy) sensation and perception. Here, we highlight the many roles precision plays in action, i.e., the key processes that rely on adequate estimates of precision, from decision making and action planning to the initiation and control of muscle movement itself. Thereby, we focus on the recent development of hierarchical, "mixed" models-generative models spanning multiple levels of discrete and continuous inference. These kinds of models open up new perspectives on the unified description of hierarchical computation, and its implementation, in action. Here, we highlight how these models reflect the many roles of precision in action-from planning to execution-and the associated pathologies if precision estimation goes wrong. We also discuss the potential biological implementation of the associated message passing, focusing on the role of neuromodulatory systems in mediating different kinds of precision.

Cortical-subcortical interactions underlie processing of auditory predictions measured with 7T fMRI.

Perception integrates both sensory inputs and internal models of the environment. In the auditory domain, predictions play a critical role because of the temporal nature of sounds. However, the precise contribution of cortical and subcortical structures in these processes and their interaction remain unclear. It is also unclear whether these brain interactions are specific to abstract rules or if they also underlie the predictive coding of local features. We used high-field 7T functional magnetic resonance imaging to investigate interactions between cortical and subcortical areas during auditory predictive processing. Volunteers listened to tone sequences in an oddball paradigm where the predictability of the deviant was manipulated. Perturbations in periodicity were also introduced to test the specificity of the response. Results indicate that both cortical and subcortical auditory structures encode high-order predictive dynamics, with the effect of predictability being strongest in the auditory cortex. These predictive dynamics were best explained by modeling a top-down information flow, in contrast to unpredicted responses. No error signals were observed to deviations of periodicity, suggesting that these responses are specific to abstract rule violations. Our results support the idea that the high-order predictive dynamics observed in subcortical areas propagate from the auditory cortex.

Effective Connectivity Network of Aberrant Prediction Error Processing in Auditory Phantom Perception.

Introduction: Prediction error (PE) is key to perception in the predictive coding framework. However, previous studies indicated the varied neural activities evoked by PE in tinnitus patients. Here, we aimed to reconcile the conflict by (1) a more nuanced view of PE, which could be driven by changing stimulus (stimulus-driven PE [sPE]) and violation of current context (context-driven PE [cPE]) and (2) investigating the aberrant connectivity networks that are engaged in the processing of the two types of PEs in tinnitus patients. Methods: Ten tinnitus patients with normal hearing and healthy controls were recruited, and a local-global auditory oddball paradigm was applied to measure the electroencephalographic difference between the two groups during sPE and cPE conditions. Results: Overall, the sPE condition engaged bottom-up and top-down connections, whereas the cPE condition engaged mostly top-down connections. The tinnitus group showed decreased sensitivity to the sPE and increased sensitivity to the cPE condition. Particularly, the auditory cortex and posterior cingulate cortex were the hubs for processing cPE in the control and tinnitus groups, respectively, showing the orientation to an internal state in tinnitus. Furthermore, tinnitus patients showed stronger connectivity to the parahippocampus and pregenual anterior cingulate cortex for the establishment of the prediction during the cPE condition. Conclusion: These results begin to dissect the role of changes in stimulus characteristics versus changes in the context of processing the same stimulus in mechanisms of tinnitus generation.

Visual experience reduces the spatial redundancy between cortical feedback inputs and primary visual cortex neurons.

The role of experience in the organization of cortical feedback (FB) remains unknown. We measured the effects of manipulating visual experience on the retinotopic specificity of supragranular and infragranular projections from the lateromedial (LM) visual area to layer (L)1 of the mouse primary visual cortex (V1). LM inputs were, on average, retinotopically matched with V1 neurons in normally and dark-reared mice, but visual exposure reduced the fraction of spatially overlapping inputs to V1. FB inputs from L5 conveyed more surround information to V1 than those from L2/3. The organization of LM inputs from L5 depended on their orientation preference and was disrupted by dark rearing. These observations were recapitulated by a model where visual experience minimizes receptive field overlap between LM inputs and V1 neurons. Our results provide a mechanism for the dependency of surround modulations on visual experience and suggest how expected interarea coactivation patterns are learned in cortical circuits.

Cortical Layer-Dependent Signaling in Cognition: Three Computational Modes of the Canonical Circuit.

The cerebral cortex performs computations via numerous six-layer modules. The operational dynamics of these modules were studied primarily in early sensory cortices using bottom-up computation for response selectivity as a model, which has been recently revolutionized by genetic approaches in mice. However, cognitive processes such as recall and imagery require top-down generative computation. The question of whether the layered module operates similarly in top-down generative processing as in bottom-up sensory processing has become testable by advances in the layer identification of recorded neurons in behaving monkeys. This review examines recent advances in laminar signaling in these two computations, using predictive coding computation as a common reference, and shows that each of these computations recruits distinct laminar circuits, particularly in layer 5, depending on the cognitive demands. These findings highlight many open questions, including how different interareal feedback pathways, originating from and terminating at different layers, convey distinct functional signals.

Collective predictive coding hypothesis: symbol emergence as decentralized Bayesian inference.

Understanding the emergence of symbol systems, especially language, requires the construction of a computational model that reproduces both the developmental learning process in everyday life and the evolutionary dynamics of symbol emergence throughout history. This study introduces the collective predictive coding (CPC) hypothesis, which emphasizes and models the interdependence between forming internal representations through physical interactions with the environment and sharing and utilizing meanings through social semiotic interactions within a symbol emergence system. The total system dynamics is theorized from the perspective of predictive coding. The hypothesis draws inspiration from computational studies grounded in probabilistic generative models and language games, including the Metropolis-Hastings naming game. Thus, playing such games among agents in a distributed manner can be interpreted as a decentralized Bayesian inference of representations shared by a multi-agent system. Moreover, this study explores the potential link between the CPC hypothesis and the free-energy principle, positing that symbol emergence adheres to the society-wide free-energy principle. Furthermore, this paper provides a new explanation for why large language models appear to possess knowledge about the world based on experience, even though they have neither sensory organs nor bodies. This paper reviews past approaches to symbol emergence systems, offers a comprehensive survey of related prior studies, and presents a discussion on CPC-based generalizations. Future challenges and potential cross-disciplinary research avenues are highlighted.

High segregation and diminished global integration in large-scale brain functional networks enhances the perceptual binding of cross-modal stimuli.

Speech perception requires the binding of spatiotemporally disjoint auditory-visual cues. The corresponding brain network-level information processing can be characterized by two complementary mechanisms: functional segregation which refers to the localization of processing in either isolated or distributed modules across the brain, and integration which pertains to cooperation among relevant functional modules. Here, we demonstrate using functional magnetic resonance imaging recordings that subjective perceptual experience of multisensory speech stimuli, real and illusory, are represented in differential states of segregation-integration. We controlled the inter-subject variability of illusory/cross-modal perception parametrically, by introducing temporal lags in the incongruent auditory-visual articulations of speech sounds within the McGurk paradigm. The states of segregation-integration balance were captured using two alternative computational approaches. First, the module responsible for cross-modal binding of sensory signals defined as the perceptual binding network (PBN) was identified using standardized parametric statistical approaches and their temporal correlations with all other brain areas were computed. With increasing illusory perception, the majority of the nodes of PBN showed decreased cooperation with the rest of the brain, reflecting states of high segregation but reduced global integration. Second, using graph theoretic measures, the altered patterns of segregation-integration were cross-validated.

Dysfunction of the magnocellular subdivision of the visual thalamus in developmental dyslexia.

Developmental dyslexia (DD) is one of the most common learning disorders, affecting millions of children and adults worldwide. To date, scientific research has attempted to explain DD primarily based on pathophysiological alterations in the cerebral cortex. In contrast, several decades ago, pioneering research on five post-mortem human brains suggested that a core characteristic of DD might be morphological alterations in a specific subdivision of the visual thalamus - the magnocellular LGN (M-LGN). However, due to considerable technical challenges in investigating LGN subdivisions non-invasively in humans, this finding was never confirmed in-vivo, and its relevance for DD pathology remained highly controversial. Here, we leveraged recent advances in high-resolution magnetic resonance imaging (MRI) at high field strength (7 Tesla) to investigate the M-LGN in DD in-vivo. Using a case-control design, we acquired data from a large sample of young adults with DD (n = 26; age 28 ± 7 years; 13 females) and matched control participants (n = 28; age 27 ± 6 years; 15 females). Each participant completed a comprehensive diagnostic behavioral test battery and participated in two MRI sessions, including three functional MRI experiments and one structural MRI acquisition. We measured blood-oxygen-level-dependent responses and longitudinal relaxation rates to compare both groups on LGN subdivision function and myelination. Based on previous research, we hypothesized that the M-LGN is altered in DD and that these alterations are associated with a key DD diagnostic score, i.e., rapid letter and number naming (RANln). The results showed aberrant responses of the M-LGN in DD compared to controls, which was reflected in a different functional lateralization of this subdivision between groups. These alterations were associated with RANln performance, specifically in male DD. We also found lateralization differences in the longitudinal relaxation rates of the M-LGN in DD relative to controls. Conversely, the other main subdivision of the LGN, the parvocellular LGN (P-LGN), showed comparable blood-oxygen-level-dependent responses and longitudinal relaxation rates between groups. The present study is the first to unequivocally show that M-LGN alterations are a hallmark of DD, affecting both the function and microstructure of this subdivision. It further provides a first functional interpretation of M-LGN alterations and a basis for a better understanding of sex-specific differences in DD with implications for prospective diagnostic and treatment strategies.

A functional contextual, observer-centric, quantum mechanical, and neuro-symbolic approach to solving the alignment problem of artificial general intelligence: safe AI through intersecting computational psychological neuroscience and LLM architecture for emergent theory of mind.

There have been impressive advancements in the field of natural language processing (NLP) in recent years, largely driven by innovations in the development of transformer-based large language models (LLM) that utilize "attention." This approach employs masked self-attention to establish (via similarly) different positions of tokens (words) within an inputted sequence of tokens to compute the most appropriate response based on its training corpus. However, there is speculation as to whether this approach alone can be scaled up to develop emergent artificial general intelligence (AGI), and whether it can address the alignment of AGI values with human values (called the alignment problem). Some researchers exploring the alignment problem highlight three aspects that AGI (or AI) requires to help resolve this problem: (1) an interpretable values specification; (2) a utility function; and (3) a dynamic contextual account of behavior. Here, a neurosymbolic model is proposed to help resolve these issues of human value alignment in AI, which expands on the transformer-based model for NLP to incorporate symbolic reasoning that may allow AGI to incorporate perspective-taking reasoning (i.e., resolving the need for a dynamic contextual account of behavior through deictics) as defined by a multilevel evolutionary and neurobiological framework into a functional contextual post-Skinnerian model of human language called "Neurobiological and Natural Selection Relational Frame Theory" (N-Frame). It is argued that this approach may also help establish a comprehensible value scheme, a utility function by expanding the expected utility equation of behavioral economics to consider functional contextualism, and even an observer (or witness) centric model for consciousness. Evolution theory, subjective quantum mechanics, and neuroscience are further aimed to help explain consciousness, and possible implementation within an LLM through correspondence to an interface as suggested by N-Frame. This argument is supported by the computational level of hypergraphs, relational density clusters, a conscious quantum level defined by QBism, and real-world applied level (human user feedback). It is argued that this approach could enable AI to achieve consciousness and develop deictic perspective-taking abilities, thereby attaining human-level self-awareness, empathy, and compassion toward others. Importantly, this consciousness hypothesis can be directly tested with a significance of approximately 5-sigma significance (with a 1 in 3.5 million probability that any identified AI-conscious observations in the form of a collapsed wave form are due to chance factors) through double-slit intent-type experimentation and visualization procedures for derived perspective-taking relational frames. Ultimately, this could provide a solution to the alignment problem and contribute to the emergence of a theory of mind (ToM) within AI.

The mechanisms of persisting disability in schizophrenia: imprecise predictive coding via cortico-striatal-thalamo-cortical loop dysfunction.

Persisting symptoms and disability remain a problem for an appreciable proportion of people with schizophrenia despite treatment with antipsychotic medication. Improving outcomes requires an understanding of the nature and mechanisms of the pathological processes underlying persistence. Classical features of schizophrenia, which include disorganization and impoverishment of mental activity, are well recognised early clinical features that predict poor long-term outcome. Substantial evidence indicates that these features reflect imprecise predictive coding. Predictive coding provides an over-arching framework for understanding efficient function of the nervous system. Imprecise predictive coding also has the potential to precipitate acute psychosis characterised by reality distortion (delusions and hallucinations) at times of stress. On the other hand, substantial evidence indicates that persistent reality distortion itself gives rise to poor occupational and social function in the long term. Furthermore, abuse of psychotomimetic drugs, which exacerbate reality distortion, contributes to poor long-term outcome in schizophrenia. Neural circuits involved in modulating volitional acts are well understood to be implicated in addiction. Plastic changes in these circuits may account for the association between psychotomimetic drug abuse and poor outcomes in schizophrenia. We propose a mechanistic model according to which unbalanced inputs to the corpus striatum disturb the precision of sub-cortical modulation of cortical activity supporting volitional action. This model accounts for the evidence that early classical symptoms predict poor outcome, while in some circumstances, persistent reality distortion also predicts poor outcome. This model has implications for the development of novel treatments that address the risk of persisting symptoms and disabilities in schizophrenia.

Adaptation of the inferior temporal neurons and efficient visual processing.

Numerous studies examining the responses of individual neurons in the inferior temporal (IT) cortex have revealed their characteristics such as two-dimensional or three-dimensional shape tuning, objects, or category selectivity. While these basic selectivities have been studied assuming that their response to stimuli is relatively stable, physiological experiments have revealed that the responsiveness of IT neurons also depends on visual experience. The activity changes of IT neurons occur over various time ranges; among these, repetition suppression (RS), in particular, is robustly observed in IT neurons without any behavioral or task constraints. I observed a similar phenomenon in the ventral visual neurons in macaque monkeys while they engaged in free viewing and actively fixated on one consistent object multiple times. This observation indicates that the phenomenon also occurs in natural situations during which the subject actively views stimuli without forced fixation, suggesting that this phenomenon is an everyday occurrence and widespread across regions of the visual system, making it a default process for visual neurons. Such short-term activity modulation may be a key to understanding the visual system; however, the circuit mechanism and the biological significance of RS remain unclear. Thus, in this review, I summarize the observed modulation types in IT neurons and the known properties of RS. Subsequently, I discuss adaptation in vision, including concepts such as efficient and predictive coding, as well as the relationship between adaptation and psychophysical aftereffects. Finally, I discuss some conceptual implications of this phenomenon as well as the circuit mechanisms and the models that may explain adaptation as a fundamental aspect of visual processing.