Amodal completion across the brain: The impact of structure and knowledge.

This study investigates the phenomenon of amodal completion within the context of naturalistic objects, employing a repetition suppression paradigm to disentangle the influence of structure and knowledge cues on how objects are completed. The research focuses on early visual cortex (EVC) and lateral occipital complex (LOC), shedding light on how these brain regions respond to different completion scenarios. In LOC, we observed suppressed responses to structure and knowledge-compatible stimuli, providing evidence that both cues influence neural processing in higher-level visual areas. However, in EVC, we did not find evidence for differential responses to completions compatible or incompatible with either structural or knowledge-based expectations. Together, our findings suggest that the interplay between structure and knowledge cues in amodal completion predominantly impacts higher-level visual processing, with less pronounced effects on the early visual cortex. This study contributes to our understanding of the complex mechanisms underlying visual perception and highlights the distinct roles played by different brain regions in amodal completion.

Perceptual confirmation bias and decision bias underlie adaptation to sequential regularities.

Our perception does not depend exclusively on the immediate sensory input. It is also influenced by our internal predictions derived from prior observations and the temporal regularities of the environment, which can result in choice history biases. However, it is unclear how this flexible use of prior information to predict the future influences perceptual decisions. Prior information may bias decisions independently of the current sensory input, or it may modulate the weight of current sensory input based on its consistency with the expectation. To address this question, we used a visual decision-making task and manipulated the transitional probabilities between successive noisy grating stimuli. Using a reverse correlation analysis, we evaluated the contribution of stimulus-independent decision bias and stimulus-dependent sensitivity modulations to choice history biases. We found that both effects coexist, whereby there was increased bias to respond in line with the predicted orientation alongside modulations in perceptual sensitivity to favor perceptual information consistent with the prediction, akin to selective attention. Furthermore, at the individual differences level, we investigated the relationship between autistic-like traits and the adaptation of choice history biases to the sequential statistics of the environment. Over two studies, we found no convincing evidence of reduced adaptation to sequential regularities in individuals with high autistic-like traits. In sum, we present robust evidence for both perceptual confirmation bias and decision bias supporting adaptation to sequential regularities in the environment.

Lexical Processing Strongly Affects Reading Times But Not Skipping During Natural Reading.

In a typical text, readers look much longer at some words than at others, even skipping many altogether. Historically, researchers explained this variation via low-level visual or oculomotor factors, but today it is primarily explained via factors determining a word's lexical processing ease, such as how well word identity can be predicted from context or discerned from parafoveal preview. While the existence of these effects is well established in controlled experiments, the relative importance of prediction, preview and low-level factors in natural reading remains unclear. Here, we address this question in three large naturalistic reading corpora (n = 104, 1.5 million words), using deep neural networks and Bayesian ideal observers to model linguistic prediction and parafoveal preview from moment to moment in natural reading. Strikingly, neither prediction nor preview was important for explaining word skipping-the vast majority of explained variation was explained by a simple oculomotor model, using just fixation position and word length. For reading times, by contrast, we found strong but independent contributions of prediction and preview, with effect sizes matching those from controlled experiments. Together, these results challenge dominant models of eye movements in reading, and instead support alternative models that describe skipping (but not reading times) as largely autonomous from word identification, and mostly determined by low-level oculomotor information.

Expected distractor context biases the attentional template for target shapes.

Visual search is supported by an internal representation of the target, the attentional template. However, which features are diagnostic of target presence critically depends on the distractors. Accordingly, previous research showed that consistent distractor context shapes the attentional template for simple targets, with the template emphasizing diagnostic dimensions (e.g., color or orientation) in blocks of trials. Here, we investigated how distractor expectations bias attentional templates for complex shapes, and tested whether such biases reflect intertrial priming or can be instantiated flexibly. Participants searched for novel shapes (cued by name) in two probabilistic distractor contexts: Either the target's orientation or rectilinearity was unique (80% validity). Across four experiments, performance was better when the distractor context was expected, indicating that target features in the expected diagnostic dimension were emphasized. Attentional templates were biased by distractor expectations when distractor context was blocked, also for participants reporting no awareness of the manipulation. Interestingly, attentional templates were also biased when distractor context was cued on a trial-by-trial basis, but only when the two contexts were consistently presented at distinct spatial locations. These results show that attentional templates can flexibly and adaptively incorporate expectations about target-distractor relations when looking for the same object in different contexts. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Humans predict the forest, not the trees: statistical learning of spatiotemporal structure in visual scenes.

The human brain is capable of using statistical regularities to predict future inputs. In the real world, such inputs typically comprise a collection of objects (e.g. a forest constitutes numerous trees). The present study aimed to investigate whether perceptual anticipation relies on lower-level or higher-level information. Specifically, we examined whether the human brain anticipates each object in a scene individually or anticipates the scene as a whole. To explore this issue, we first trained participants to associate co-occurring objects within fixed spatial arrangements. Meanwhile, participants implicitly learned temporal regularities between these displays. We then tested how spatial and temporal violations of the structure modulated behavior and neural activity in the visual system using fMRI. We found that participants only showed a behavioral advantage of temporal regularities when the displays conformed to their previously learned spatial structure, demonstrating that humans form configuration-specific temporal expectations instead of predicting individual objects. Similarly, we found suppression of neural responses for temporally expected compared with temporally unexpected objects in lateral occipital cortex only when the objects were embedded within expected configurations. Overall, our findings indicate that humans form expectations about object configurations, demonstrating the prioritization of higher-level over lower-level information in temporal expectation.

Stubborn Predictions in Primary Visual Cortex.

Perceivers can use past experiences to make sense of ambiguous sensory signals. However, this may be inappropriate when the world changes and past experiences no longer predict what the future holds. Optimal learning models propose that observers decide whether to stick with or update their predictions by tracking the uncertainty or "precision" of their expectations. However, contrasting theories of prediction have argued that we are prone to misestimate uncertainty-leading to stubborn predictions that are difficult to dislodge. To compare these possibilities, we had participants learn novel perceptual predictions before using fMRI to record visual brain activity when predictive contingencies were disrupted-meaning that previously "expected" events became objectively improbable. Multivariate pattern analyses revealed that expected events continued to be decoded with greater fidelity from primary visual cortex, despite marked changes in the statistical structure of the environment, which rendered these expectations no longer valid. These results suggest that our perceptual systems do indeed form stubborn predictions even from short periods of learning-and more generally suggest that top-down expectations have the potential to help or hinder perceptual inference in bounded minds like ours.

Conceptual Associations Generate Sensory Predictions.

A crucial ability of the human brain is to learn and exploit probabilistic associations between stimuli to facilitate perception and behavior by predicting future events. Although studies have shown how perceptual relationships are used to predict sensory inputs, relational knowledge is often between concepts rather than percepts (e.g., we learned to associate cats with dogs, rather than specific images of cats and dogs). Here, we asked if and how sensory responses to visual input may be modulated by predictions derived from conceptual associations. To this end we exposed participants of both sexes to arbitrary word-word pairs (e.g., car-dog) repeatedly, creating an expectation of the second word, conditional on the occurrence of the first. In a subsequent session, we exposed participants to novel word-picture pairs, while measuring fMRI BOLD responses. All word-picture pairs were equally likely, but half of the pairs conformed to the previously formed conceptual (word-word) associations, whereas the other half violated this association. Results showed suppressed sensory responses throughout the ventral visual stream, including early visual cortex, to pictures that corresponded to the previously expected words compared with unexpected words. This suggests that the learned conceptual associations were used to generate sensory predictions that modulated processing of the picture stimuli. Moreover, these modulations were tuning specific, selectively suppressing neural populations tuned toward the expected input. Combined, our results suggest that recently acquired conceptual priors are generalized across domains and used by the sensory brain to generate category-specific predictions, facilitating processing of expected visual input.SIGNIFICANCE STATEMENT Perceptual predictions play a crucial role in facilitating perception and the integration of sensory information. However, little is known about whether and how the brain uses more abstract, conceptual priors to form sensory predictions. In our preregistered study, we show that priors derived from recently acquired arbitrary conceptual associations result in category-specific predictions that modulate perceptual processing throughout the ventral visual hierarchy, including early visual cortex. These results suggest that the predictive brain uses prior knowledge across various domains to modulate perception, thereby extending our understanding of the extensive role predictions play in perception.

An adversarial collaboration protocol for testing contrasting predictions of global neuronal workspace and integrated information theory.

The relationship between conscious experience and brain activity has intrigued scientists and philosophers for centuries. In the last decades, several theories have suggested different accounts for these relationships. These theories have developed in parallel, with little to no cross-talk among them. To advance research on consciousness, we established an adversarial collaboration between proponents of two of the major theories in the field, Global Neuronal Workspace and Integrated Information Theory. Together, we devised and preregistered two experiments that test contrasting predictions of these theories concerning the location and timing of correlates of visual consciousness, which have been endorsed by the theories' proponents. Predicted outcomes should either support, refute, or challenge these theories. Six theory-impartial laboratories will follow the study protocol specified here, using three complementary methods: Functional Magnetic Resonance Imaging (fMRI), Magneto-Electroencephalography (M-EEG), and intracranial electroencephalography (iEEG). The study protocol will include built-in replications, both between labs and within datasets. Through this ambitious undertaking, we hope to provide decisive evidence in favor or against the two theories and clarify the footprints of conscious visual perception in the human brain, while also providing an innovative model of large-scale, collaborative, and open science practice.

Successor-like representation guides the prediction of future events in human visual cortex and hippocampus.

Human agents build models of their environment, which enable them to anticipate and plan upcoming events. However, little is known about the properties of such predictive models. Recently, it has been proposed that hippocampal representations take the form of a predictive map-like structure, the so-called successor representation (SR). Here, we used human functional magnetic resonance imaging to probe whether activity in the early visual cortex (V1) and hippocampus adhere to the postulated properties of the SR after visual sequence learning. Participants were exposed to an arbitrary spatiotemporal sequence consisting of four items (A-B-C-D). We found that after repeated exposure to the sequence, merely presenting single sequence items (e.g., - B - -) resulted in V1 activation at the successor locations of the full sequence (e.g., C-D), but not at the predecessor locations (e.g., A). This highlights that visual representations are skewed toward future states, in line with the SR. Similar results were also found in the hippocampus. Moreover, the hippocampus developed a coactivation profile that showed sensitivity to the temporal distance in sequence space, with fading representations for sequence events in the more distant past and future. V1, in contrast, showed a coactivation profile that was only sensitive to spatial distance in stimulus space. Taken together, these results provide empirical evidence for the proposition that both visual and hippocampal cortex represent a predictive map of the visual world akin to the SR.

Magnetoencephalography recordings reveal the neural mechanisms of auditory contributions to improved visual detection.

Sounds enhance the detection of visual stimuli while concurrently biasing an observer's decisions. To investigate the neural mechanisms that underlie such multisensory interactions, we decoded time-resolved Signal Detection Theory sensitivity and criterion parameters from magneto-encephalographic recordings of participants that performed a visual detection task. We found that sounds improved visual detection sensitivity by enhancing the accumulation and maintenance of perceptual evidence over time. Meanwhile, criterion decoding analyses revealed that sounds induced brain activity patterns that resembled the patterns evoked by an actual visual stimulus. These two complementary mechanisms of audiovisual interplay differed in terms of their automaticity: Whereas the sound-induced enhancement in visual sensitivity depended on participants being actively engaged in a detection task, we found that sounds activated the visual cortex irrespective of task demands, potentially inducing visual illusory percepts. These results challenge the classical assumption that sound-induced increases in false alarms exclusively correspond to decision-level biases.

Choice Boosts Curiosity.

In our connected era, we spend significant time and effort satisfying our curiosity. Often, we choose which information we seek, but sometimes the selection is made for us. We hypothesized that humans exhibit enhanced curiosity in the context of choice. We designed a task in which healthy participants saw two lotteries on each trial. On some trials, participants chose which lottery to play. On other trials, the lottery was selected for them. Participants then indicated their curiosity about the outcome of the to-be-played lottery via self-report ratings (Experiment 1, N = 34) or willingness-to-wait decisions (Experiment 2, N = 34). We found that participants exhibited higher curiosity ratings and greater willingness to wait for the outcome of lotteries they had chosen than for lotteries that had been selected for them (controlling for initial preference). This demonstrates that choice boosts curiosity, which may have implications for boosting learning, memory, and motivation.

Cortical activity during naturalistic music listening reflects short-range predictions based on long-term experience.

Expectations shape our experience of music. However, the internal model upon which listeners form melodic expectations is still debated. Do expectations stem from Gestalt-like principles or statistical learning? If the latter, does long-term experience play an important role, or are short-term regularities sufficient? And finally, what length of context informs contextual expectations? To answer these questions, we presented human listeners with diverse naturalistic compositions from Western classical music, while recording neural activity using MEG. We quantified note-level melodic surprise and uncertainty using various computational models of music, including a state-of-the-art transformer neural network. A time-resolved regression analysis revealed that neural activity over fronto-temporal sensors tracked melodic surprise particularly around 200ms and 300-500ms after note onset. This neural surprise response was dissociated from sensory-acoustic and adaptation effects. Neural surprise was best predicted by computational models that incorporated long-term statistical learning-rather than by simple, Gestalt-like principles. Yet, intriguingly, the surprise reflected primarily short-range musical contexts of less than ten notes. We present a full replication of our novel MEG results in an openly available EEG dataset. Together, these results elucidate the internal model that shapes melodic predictions during naturalistic music listening.

Updating Contextual Sensory Expectations for Adaptive Behavior.

The brain has the extraordinary capacity to construct predictive models of the environment by internalizing statistical regularities in the sensory inputs. The resulting sensory expectations shape how we perceive and react to the world; at the neural level, this relates to decreased neural responses to expected than unexpected stimuli ("expectation suppression"). Crucially, expectations may need revision as context changes. However, existing research has often neglected this issue. Further, it is unclear whether contextual revisions apply selectively to expectations relevant to the task at hand, hence serving adaptive behavior. The present fMRI study examined how contextual visual expectations spread throughout the cortical hierarchy as we update our beliefs. We created a volatile environment: two alternating contexts contained different sequences of object images, thereby producing context-dependent expectations that needed revision when the context changed. Human participants of both sexes attended a training session before scanning to learn the contextual sequences. The fMRI experiment then tested for the emergence of contextual expectation suppression in two separate tasks, respectively, with task-relevant and task-irrelevant expectations. Effects of contextual expectation emerged progressively across the cortical hierarchy as participants attuned themselves to the context: expectation suppression appeared first in the insula, inferior frontal gyrus, and posterior parietal cortex, followed by the ventral visual stream, up to early visual cortex. This applied selectively to task-relevant expectations. Together, the present results suggest that an insular and frontoparietal executive control network may guide the flexible deployment of contextual sensory expectations for adaptive behavior in our complex and dynamic world.SIGNIFICANCE STATEMENT The world is structured by statistical regularities, which we use to predict the future. This is often accompanied by suppressed neural responses to expected compared with unexpected events ("expectation suppression"). Crucially, the world is also highly volatile and context-dependent: expected events may become unexpected when the context changes, thus raising the crucial need for belief updating. However, this issue has generally been neglected. By setting up a volatile environment, we show that expectation suppression emerges first in executive control regions, followed by relevant sensory areas, only when observers use their expectations to optimize behavior. This provides surprising yet clear evidence on how the brain controls the updating of sensory expectations for adaptive behavior in our ever-changing world.

Reconstructing the predictive architecture of the mind and brain.

Predictive processing has become an influential framework in cognitive neuroscience. However, it often lacks specificity and direct empirical support. How can we probe the nature and limits of the predictive brain? We highlight the potential of recent advances in artificial intelligence (AI) for providing a richer and more computationally explicit test of this theory of cortical function.

A hierarchy of linguistic predictions during natural language comprehension.

Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction to guide the interpretation of incoming input. However, the role of prediction in language processing remains disputed, with disagreement about both the ubiquity and representational nature of predictions. Here, we address both issues by analyzing brain recordings of participants listening to audiobooks, and using a deep neural network (GPT-2) to precisely quantify contextual predictions. First, we establish that brain responses to words are modulated by ubiquitous predictions. Next, we disentangle model-based predictions into distinct dimensions, revealing dissociable neural signatures of predictions about syntactic category (parts of speech), phonemes, and semantics. Finally, we show that high-level (word) predictions inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore the ubiquity of prediction in language processing, showing that the brain spontaneously predicts upcoming language at multiple levels of abstraction.

Flexible recoding of visual input for memory storage.

Working memory enables us to maintain relevant past information for future behavior. In this issue of Neuron, Kwak and Curtis (2022) demonstrate that early visual areas do not simply maintain but flexibly recode sensory percepts into mnemonic codes containing goal-relevant information.

Acute threat enhances perceptual sensitivity without affecting the decision criterion.

Threatening situations ask for rapid and accurate perceptual decisions to optimize coping. Theoretical models have stated that psychophysiological states, such as bradycardia during threat-anticipatory freezing, may facilitate perception. However, it's unclear if this occurs via enhanced bottom-up sensory processing or by relying more on prior expectations. To test this, 52 (26 female) participants completed a visual target-detection paradigm under threat-of-shock (15% reinforcement rate) with a manipulation of prior expectations. Participants judged the presence of a backward-masked grating (target presence rate 50%) after systematically manipulating their decision criterion with a rare (20%) or frequent (80%) target presence rate procedure. Threat-of-shock induced stronger heart rate deceleration compared to safe, indicative of threat-anticipatory freezing. Importantly, threat-of-shock enhanced perceptual sensitivity but we did not find evidence of an altered influence of the effect of prior expectations on current decisions. Correct target detection (hits) was furthermore accompanied by an increase in the magnitude of this heart rate deceleration compared to a missed target. While this was independent of threat-of-shock manipulation, only under threat-of-shock this increase was accompanied by more hits and increased sensitivity. Together, these findings suggest that under acute threat participants may rely more on bottom-up sensory processing versus prior expectations in perceptual decision-making. Critically, bradycardia may underlie such enhanced perceptual sensitivity.

Adaptation and serial choice bias for low-level visual features are unaltered in autistic adolescents.

Autism spectrum disorder (ASD), or autism, is characterized by social and non-social symptoms, including sensory hyper- and hyposensitivities. A suggestion has been put forward that some of these symptoms could be explained by differences in how sensory information is integrated with its context, including a lower tendency to leverage the past in the processing of new perceptual input. At least two history-dependent effects of opposite directions have been described in the visual perception literature: a repulsive adaptation effect, where perception of a stimulus is biased away from an adaptor stimulus, and an attractive serial choice bias, where perceptual choices are biased toward the previous choice. In this study, we investigated whether autistic participants differed in either bias from typically developing controls (TDs). Sixty-four adolescent participants (31 with ASD, 33 TDs) were asked to categorize oriented line stimuli in two tasks that were designed so that we would induce either adaptation or serial choice bias. Although our tasks successfully induced both biases, in comparing the two groups we found no differences in the magnitude of adaptation nor in the modulation of perceptual choices by the previous choice. In conclusion, we find no evidence of a decreased integration of the past in visual perception of low-level stimulus features in autistic adolescents.

Brief Stimuli Cast a Persistent Long-Term Trace in Visual Cortex.

Visual processing is strongly influenced by recent stimulus history, a phenomenon termed adaptation. Prominent theories cast adaptation as a consequence of optimized encoding of visual information by exploiting the temporal statistics of the world. However, this would require the visual system to track the history of individual briefly experienced events, within a stream of visual input, to build up statistical representations over longer timescales. Here, using an openly available dataset from the Allen Brain Observatory, we show that neurons in the early visual cortex of the mouse indeed maintain long-term traces of individual past stimuli that persist despite the presentation of several intervening stimuli, leading to long-term and stimulus-specific adaptation over dozens of seconds. Long-term adaptation was selectively expressed in cortical, but not in thalamic, neurons, which only showed short-term adaptation. Early visual cortex thus maintains concurrent stimulus-specific memory traces of past input, enabling the visual system to build up a statistical representation of the world to optimize the encoding of new information in a changing environment.SIGNIFICANCE STATEMENT In the natural world, previous sensory input is predictive of current input over multisecond timescales. The visual system could exploit these predictabilities by adapting current visual processing to the long-term history of visual input. However, it is unclear whether the visual system can track the history of individual briefly experienced images, within a stream of input, to build up statistical representations over such long timescales. Here, we show that neurons in early visual cortex of the mouse brain exhibit remarkably long-term adaptation to brief stimuli, persisting over dozens of seconds, and despite the presentation of several intervening stimuli. The visual cortex thus maintains long-term traces of individual briefly experienced past images, enabling the formation of statistical representations over extended timescales.

Scene Context Impairs Perception of Semantically Congruent Objects.

Visual scene context is well-known to facilitate the recognition of scene-congruent objects. Interestingly, however, according to predictive-processing accounts of brain function, scene congruency may lead to reduced (rather than enhanced) processing of congruent objects, compared with incongruent ones, because congruent objects elicit reduced prediction-error responses. We tested this counterintuitive hypothesis in two online behavioral experiments with human participants (N = 300). We found clear evidence for impaired perception of congruent objects, both in a change-detection task measuring response times and in a bias-free object-discrimination task measuring accuracy. Congruency costs were related to independent subjective congruency ratings. Finally, we show that the reported effects cannot be explained by low-level stimulus confounds, response biases, or top-down strategy. These results provide convincing evidence for perceptual congruency costs during scene viewing, in line with predictive-processing theory.