The Influence of Memory on Visual Perception in Infants, Children, and Adults.

Perception is not an independent, in-the-moment event. Instead, perceiving involves integrating prior expectations with current observations. How does this ability develop from infancy through adulthood? We examined how prior visual experience shapes visual perception in infants, children, and adults. Using an identical task across age groups, we exposed participants to pairs of colorful stimuli and implicitly measured their ability to discriminate relative saturation levels. Results showed that adult participants were biased by previously experienced exemplars, and exhibited weakened in-the-moment discrimination between different levels of saturation. In contrast, infants and children showed less influence of memory in their perception, and they actually outperformed adults in discriminating between current levels of saturation. Our findings suggest that as humans develop, their perception relies more on prior experience and less on current observation.

iCatcher+: Robust and Automated Annotation of Infants' and Young Children's Gaze Behavior From Videos Collected in Laboratory, Field, and Online Studies.

Technological advances in psychological research have enabled large-scale studies of human behavior and streamlined pipelines for automatic processing of data. However, studies of infants and children have not fully reaped these benefits because the behaviors of interest, such as gaze duration and direction, still have to be extracted from video through a laborious process of manual annotation, even when these data are collected online. Recent advances in computer vision raise the possibility of automated annotation of these video data. In this article, we built on a system for automatic gaze annotation in young children, iCatcher, by engineering improvements and then training and testing the system (referred to hereafter as iCatcher+) on three data sets with substantial video and participant variability (214 videos collected in U.S. lab and field sites, 143 videos collected in Senegal field sites, and 265 videos collected via webcams in homes; participant age range = 4 months-3.5 years). When trained on each of these data sets, iCatcher+ performed with near human-level accuracy on held-out videos on distinguishing "LEFT" versus "RIGHT" and "ON" versus "OFF" looking behavior across all data sets. This high performance was achieved at the level of individual frames, experimental trials, and study videos; held across participant demographics (e.g., age, race/ethnicity), participant behavior (e.g., movement, head position), and video characteristics (e.g., luminance); and generalized to a fourth, entirely held-out online data set. We close by discussing next steps required to fully automate the life cycle of online infant and child behavioral studies, representing a key step toward enabling robust and high-throughput developmental research.

iCatcher: A neural network approach for automated coding of young children's eye movements.

Infants' looking behaviors are often used for measuring attention, real-time processing, and learning-often using low-resolution videos. Despite the ubiquity of gaze-related methods in developmental science, current analysis techniques usually involve laborious post hoc coding, imprecise real-time coding, or expensive eye trackers that may increase data loss and require a calibration phase. As an alternative, we propose using computer vision methods to perform automatic gaze estimation from low-resolution videos. At the core of our approach is a neural network that classifies gaze directions in real time. We compared our method, called iCatcher, to manually annotated videos from a prior study in which infants looked at one of two pictures on a screen. We demonstrated that the accuracy of iCatcher approximates that of human annotators and that it replicates the prior study's results. Our method is publicly available as an open-source repository at https://github.com/yoterel/iCatcher.

Temporal Predictability Modulates Cortical Activity and Functional Connectivity in the Frontoparietal Network in 6-Month-Old Infants.

Despite the abundance of behavioral evidence showing the interaction between attention and prediction in infants, the neural underpinnings of this interaction are not yet well understood. The endogenous attentional function in adults have been largely localized to the frontoparietal network. However, resting-state and neuroanatomical investigations have found that this frontoparietal network exhibits a protracted developmental trajectory and involves weak and unmyelinated long-range connections early in infancy. Can this developmentally nascent network still be modulated by predictions? Here, we conducted the first investigation of infant frontoparietal network engagement as a function of the predictability of visual events. Using functional near-infrared spectroscopy, the hemodynamic response in the frontal, parietal, and occipital lobes was analyzed as infants watched videos of temporally predictable or unpredictable sequences. We replicated previous findings of cortical signal attenuation in the frontal and sensory cortices in response to predictable sequences and extended these findings to the parietal lobe. We also estimated background functional connectivity (i.e., by regressing out task-evoked responses) to reveal that frontoparietal functional connectivity was significantly greater during predictable sequences compared to unpredictable sequences, suggesting that this frontoparietal network may underlie how the infant brain communicates predictions. Taken together, our results illustrate that temporal predictability modulates the activation and connectivity of the frontoparietal network early in infancy, supporting the notion that this network may be functionally available early in life despite its protracted developmental trajectory.

Perceptual inference is impaired in individuals with ASD and intact in individuals who have lost the autism diagnosis.

Beyond the symptoms which characterize their diagnoses, individuals with autism spectrum disorder (ASD) show enhanced performance in simple perceptual discrimination tasks. Often attributed to superior sensory sensitivities, enhanced performance may also reflect a weaker bias towards previously perceived stimuli. This study probes perceptual inference in a group of individuals who have lost the autism diagnosis (LAD); that is, they were diagnosed with ASD in early childhood but have no current ASD symptoms. Groups of LAD, current ASD, and typically developing (TD) participants completed an auditory discrimination task. Individuals with TD showed a bias towards previously perceived stimuli-a perceptual process called "contraction bias"; that is, their representation of a given tone was contracted towards the preceding trial stimulus in a manner that is Bayesian optimal. Similarly, individuals in the LAD group showed a contraction bias. In contrast, individuals with current ASD showed a weaker contraction bias, suggesting reduced perceptual inferencing. These findings suggest that changes that characterize LAD extend beyond the social and communicative symptoms of ASD, impacting perceptual domains. Measuring perceptual processing earlier in development in ASD will tap the causality between changes in perceptual and symptomatological domains. Further, the characterization of perceptual inference could reveal meaningful individual differences in complex high-level behaviors.

How an infant's active response to structured experience supports perceptual-cognitive development.

Previous research on perceptual and cognitive development has predominantly focused on infants' passive response to experience. For example, if infants are exposed to acoustic patterns in the background while they are engaged in another activity, what are they able to learn? However, recent work in this area has revealed that even very young infants are also capable of active perceptual and cognitive responses to experience. Specifically, recent neuroimaging work showed that infants' perceptual systems predict upcoming sensory events and that learning to predict new events rapidly modulates the responses of their perceptual systems. In addition, there is new evidence that young infants have access to endogenous attention and their prediction and attention are rapidly and robustly modified through learning about patterns in the environment. In this chapter, we present a synthesis of the existing research on the impact of infants' active responses to experience and argue that this active engagement importantly supports infants' perceptual-cognitive development. To this end, we first define what a mechanism of active engagement is and examine how learning, selective attention, and prediction can be considered active mechanisms. Then, we argue that these active mechanisms become engaged in response to higher-order environmental structures, such as temporal, spatial, and relational patterns, and review both behavioral and neural evidence of infants' active responses to these structures or patterns. Finally, we discuss how this active engagement in infancy may give rise to the emergence of specialized perceptual-cognitive systems and highlight directions for future research.

Video-based motion-resilient reconstruction of three-dimensional position for functional near-infrared spectroscopy and electroencephalography head mounted probes.

Significance: We propose a video-based, motion-resilient, and fast method for estimating the position of optodes on the scalp. Aim: Measuring the exact placement of probes (e.g., electrodes and optodes) on a participant's head is a notoriously difficult step in acquiring neuroimaging data from methods that rely on scalp recordings (e.g., electroencephalography and functional near-infrared spectroscopy) and is particularly difficult for any clinical or developmental population. Existing methods of head measurements require the participant to remain still for a lengthy period of time, are laborious, and require extensive training. Therefore, a fast and motion-resilient method is required for estimating the scalp location of probes. Approach: We propose an innovative video-based method for estimating the probes' positions relative to the participant's head, which is fast, motion-resilient, and automatic. Our method builds on capitalizing the advantages and understanding the limitations of cutting-edge computer vision and machine learning tools. We validate our method on 10 adult subjects and provide proof of feasibility with infant subjects. Results: We show that our method is both reliable and valid compared to existing state-of-the-art methods by estimating probe positions in a single measurement and by tracking their translation and consistency across sessions. Finally, we show that our automatic method is able to estimate the position of probes on an infant head without lengthy offline procedures, a task that has been considered challenging until now. Conclusions: Our proposed method allows, for the first time, the use of automated spatial co-registration methods on developmental and clinical populations, where lengthy, motion-sensitive measurement methods routinely fail.

Memory integration into visual perception in infancy, childhood, and adulthood.

We compared the influence of prior knowledge on visual perception in infants, children, and adults in order to explore the developmental trajectory by which prior knowledge is integrated with new sensory input. Using an identical task across age groups, we tested how participants' accumulated experience affected their ability to judge the relative saturation levels within a pair of sequentially-presented stimuli. We found that infants and children, relative to adults, showed greater influence of the current observation and reduced influence of memory in their perception. In fact, infants and children outperformed adults in discriminating between different levels of saturation, and their performance was less biased by previously-experienced exemplars. Thus, the development of perceptual integration of memory leads to less precise discrimination in the moment, but allows observers to make use of their prior experience in interpreting a complex sensory environment.

A Computational Role for Top-Down Modulation from Frontal Cortex in Infancy.

Recent findings have shown that full-term infants engage in top-down sensory prediction, and these predictions are impaired as a result of premature birth. Here, we use an associative learning model to uncover the neuroanatomical origins and computational nature of this top-down signal. Infants were exposed to a probabilistic audiovisual association. We find that both groups (full term, preterm) have a comparable stimulus-related response in sensory and frontal lobes and track prediction error in their frontal lobes. However, preterm infants differ from their full-term peers in weaker tracking of prediction error in sensory regions. We infer that top-down signals from the frontal lobe to the sensory regions carry information about prediction error. Using computational learning models and comparing neuroimaging results from full-term and preterm infants, we have uncovered the computational content of top-down signals in young infants when they are engaged in a probabilistic associative learning.

Auditory frequency discrimination is correlated with linguistic skills, but its training does not improve them or other pitch discrimination tasks.

Brain-training, aimed at advancing and improving cognitive and perceptual abilities, is vastly studied because of its immense promise. Yet, there are major controversies regarding its main claim that intensive weeks' training on a single challenging task could improve performance in related untrained tasks. Ample training studies showing transfer were criticized for flawed design. We now explored the impact of perceptual training (auditory frequency discrimination), applying a carefully controlled intensive training experiment. First, we administered a battery of perceptual, linguistic, and cognitive tasks to a large population to determine "near" to "far" tasks according to (pretraining) correlations in performance. This assessment revealed significant correlations between simple pitch discrimination and complex linguistic tasks, including reading and syntactic reasoning. Second, we administered a broad test battery before (and after) training, which included several tasks assessing pitch discrimination, and the linguistic tasks that showed pretraining correlation with auditory frequency discrimination. The test group trained with 2 tone frequency discrimination for 40 sessions. An active control group trained with a working memory (n-back) task for the same duration, and a passive control group was only tested before and after training. Pretraining performance levels were similar in the three groups. Our results were straightforward. No transfer was found to untrained tasks that rely on pith discrimination, or to linguistic tasks that showed pretraining correlation. Mild to marginal transfer was found only to pitch discrimination tasks using almost exactly the trained protocol. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Perceptual bias reveals slow-updating in autism and fast-forgetting in dyslexia.

Individuals with autism and individuals with dyslexia both show reduced use of previous sensory information (stimuli statistics) in perceptual tasks, even though these are very different neurodevelopmental disorders. To better understand how past sensory information influences the perceptual experience in these disorders, we first investigated the trial-by-trial performance of neurotypical participants in a serial discrimination task. Neurotypical participants overweighted recent stimuli, revealing fast updating of internal sensory models, which is adaptive in changing environments. They also weighted the detailed stimuli distribution inferred by longer-term accumulation of stimuli statistics, which is adaptive in stable environments. Compared to neurotypical participants, individuals with dyslexia weighted earlier stimuli less heavily, whereas individuals with autism spectrum disorder weighted recent stimuli less heavily. Investigating the dynamics of perceptual inference reveals that individuals with dyslexia rely more on information about the immediate past, whereas perception in individuals with autism is dominated by longer-term statistics.

Prediction in infants and adults: A pupillometry study.

Adults use both bottom-up sensory inputs and top-down signals to generate predictions about future sensory inputs. Infants have also been shown to make predictions with simple stimuli and recent work has suggested top-down processing is available early in infancy. However, it is unknown whether this indicates that top-down prediction is an ability that is continuous across the lifespan or whether an infant's ability to predict is different from an adult's, qualitatively or quantitatively. We employed pupillometry to provide a direct comparison of prediction abilities across these disparate age groups. Pupil dilation response (PDR) was measured in 6-month olds and adults as they completed an identical implicit learning task designed to help learn associations between sounds and pictures. We found significantly larger PDR for visual omission trials (i.e. trials that violated participants' predictions without the presentation of new stimuli to control for bottom-up signals) compared to visual present trials (i.e. trials that confirmed participants' predictions) in both age groups. Furthermore, a computational learning model that is closely linked to prediction error (Rescorla-Wagner model) demonstrated similar learning trajectories suggesting a continuity of predictive capacity and learning across the two age groups.

Shorter cortical adaptation in dyslexia is broadly distributed in the superior temporal lobe and includes the primary auditory cortex.

Studies of the performance of individuals with dyslexia in perceptual tasks suggest that their implicit inference of sound statistics is impaired. Previously, using two-tone frequency discrimination, we found that the effect of previous trials' frequencies on the judgments of individuals with dyslexia decays faster than the effect on controls' judgments, and that the adaptation (decrease of neural response to repeated stimuli) of their ERP responses to tones is shorter (Jaffe-Dax et al., 2017). Here, we show the cortical distribution of these abnormal dynamics of adaptation using fast-acquisition fMRI. We find that faster decay of adaptation in dyslexia is widespread, although the most significant effects are found in the left superior temporal lobe, including the auditory cortex. This broad distribution suggests that the faster decay of implicit memory of individuals with dyslexia is a general characteristic of their cortical dynamics, which also affects sensory cortices.

Dyslexics' faster decay of implicit memory for sounds and words is manifested in their shorter neural adaptation.

Dyslexia is a prevalent reading disability whose underlying mechanisms are still disputed. We studied the neural mechanisms underlying dyslexia using a simple frequency-discrimination task. Though participants were asked to compare the two tones in each trial, implicit memory of previous trials affected their responses. We hypothesized that implicit memory decays faster among dyslexics. We tested this by increasing the temporal intervals between consecutive trials, and by measuring the behavioral impact and ERP responses from the auditory cortex. Dyslexics showed a faster decay of implicit memory effects on both measures, with similar time constants. Finally, faster decay of implicit memory also characterized the impact of sound regularities in benefitting dyslexics' oral reading rate. Their benefit decreased faster as a function of the time interval from the previous reading of the same non-word. We propose that dyslexics' shorter neural adaptation paradoxically accounts for their longer reading times, since it reduces their temporal window of integration of past stimuli, resulting in noisier and less reliable predictions for both simple and complex stimuli. Less reliable predictions limit their acquisition of reading expertise.

Dyslexics' usage of visual priors is impaired.

Human perception benefits substantially from familiarity, via the formation of effective predictions of the environment's pattern of stimulation. Basic stimulation characteristics are automatically retrieved and integrated into our perception. A quantitatively measurable manifestation of the integration of priors is known as "contraction to the mean"; i.e., perception is biased toward the experienced mean. We previously showed that in the context of auditory discrimination, the magnitude of this bias is smaller among dyslexic individuals than among good readers matched for age and general reasoning skills. Here we examined whether a similarly reduced contraction characterizes dyslexics' behavior on serial visual tasks. Using serial spatial frequency discrimination tasks, we found that dyslexics' bias toward the experiment's mean spatial frequency was smaller than that observed for the controls. Thus, dyslexics' difficulties in automatic detection and integration of stimulus statistics are domain-general. These difficulties are likely to impede the acquisition of reading expertise.

A Computational Model of Implicit Memory Captures Dyslexics' Perceptual Deficits.

Dyslexics are diagnosed for their poor reading skills, yet they characteristically also suffer from poor verbal memory and often from poor auditory skills. To date, this combined profile has been accounted for in broad cognitive terms. Here we hypothesize that the perceptual deficits associated with dyslexia can be understood computationally as a deficit in integrating prior information with noisy observations. To test this hypothesis we analyzed the performance of human participants in an auditory discrimination task using a two-parameter computational model. One parameter captures the internal noise in representing the current event, and the other captures the impact of recently acquired prior information. Our findings show that dyslexics' perceptual deficit can be accounted for by inadequate adjustment of these components; namely, low weighting of their implicit memory of past trials relative to their internal noise. Underweighting the stimulus statistics decreased dyslexics' ability to compensate for noisy observations. ERP measurements (P2 component) while participants watched a silent movie indicated that dyslexics' perceptual deficiency may stem from poor automatic integration of stimulus statistics. This study provides the first description of a specific computational deficit associated with dyslexia. SIGNIFICANCE STATEMENT: This study presents the first attempt to specify the mechanisms underlying dyslexics' perceptual difficulties computationally by applying a specific model, inspired by the Bayesian framework. This model dissociates between the contribution of sensory noise and that of the prior statistics in an auditory perceptual decision task. We show that dyslexics cannot compensate for their perceptual noise by incorporating prior information. By contrast, adequately reading controls' usage of previous information is often close to optimal. We used ERP measurements to assess the neuronal stage of this deficit. We found that unlike their peers, dyslexics' ERP responses are not sensitive to the relations between the current observation and the prior observation, indicating that they cannot establish a reliable prior.