Frontal-to-visual information flow explains predictive motion tracking.

Predictive tracking demonstrates our ability to maintain a line of vision on moving objects even when they temporarily disappear. Models of smooth pursuit eye movements posit that our brain achieves this ability by directly streamlining motor programming from continuously updated sensory motion information. To test this hypothesis, we obtained sensory motion representation from multivariate electroencephalogram activity while human participants covertly tracked a temporarily occluded moving stimulus with their eyes remaining stationary at the fixation point. The sensory motion representation of the occluded target evolves to its maximum strength at the expected timing of reappearance, suggesting a timely modulation of the internal model of the visual target. We further characterize the spatiotemporal dynamics of the task-relevant motion information by computing the phase gradients of slow oscillations. We discovered a predominant posterior-to-anterior phase gradient immediately after stimulus occlusion; however, at the expected timing of reappearance, the axis reverses the gradient, becoming anterior-to-posterior. The behavioral bias of smooth pursuit eye movements, which is a signature of the predictive process of the pursuit, was correlated with the posterior division of the gradient. These results suggest that the sensory motion area modulated by the prediction signal is involved in updating motor programming.

The Representational Dynamics of Sequential Perceptual Averaging.

It is clear that humans can extract statistical information from streams of visual input, yet how our brain processes sequential images into the abstract representation of the mean feature value remains poorly explored. Using multivariate pattern analyses of electroencephalography recorded while human observers viewed 10 sequentially presented Gabors of different orientations to estimate their mean orientation at the end, we investigated sequential averaging mechanism by tracking the quality of individual and mean orientation as a function of sequential position. Critically, we varied the sequential variance of Gabor orientations to understand the neural basis of perceptual mean errors occurring during a sequential averaging task. We found that the mean-orientation representation emerged at specific delays from each sequential stimulus onset and became increasingly accurate as additional Gabors were viewed. Especially in frontocentral electrodes, the neural representation of mean orientation improved more rapidly and to a greater degree in less volatile environments, whereas individual orientation information was encoded precisely regardless of environmental volatility. The computational analysis of behavioral data also showed that perceptual mean errors arise from the cumulative construction of the mean orientation rather than the low-level encoding of individual stimulus orientation. Thus, our findings provide neural mechanisms to differentially accumulate increasingly abstract features from a concrete piece of information across the cortical hierarchy depending on environmental volatility.SIGNIFICANCE STATEMENT The visual system extracts behaviorally relevant summary statistical representation by exploiting statistical regularity of the visual stream over time. However, how the neural representation of the abstract mean feature value develops in a temporally changing environment remains poorly identified. Here, we directly recover the mean orientation information of sequentially delivered Gabor stimuli with different orientations as a function of their positions in time. The mean orientation representation, which is regularly updated, becomes increasingly accurate with increasing sequential position especially in the frontocentral region. Further, perceptual mean errors arise from the cumulative process rather than the low-level stimulus encoding. Overall, our study reveals a role of higher cortical areas in integrating stimulus-specific information into increasingly abstract task-oriented information.

Induced astigmatism biases the orientation information represented in multivariate electroencephalogram activities.

A small physical change in the eye influences the entire neural information process along the visual pathway, causing perceptual errors and behavioral changes. Astigmatism, a refractive error in which visual images do not evenly focus on the retina, modulates visual perception, and the accompanying neural processes in the brain. However, studies on the neural representation of visual stimuli in astigmatism are scarce. We investigated the relationship between retinal input distortions and neural bias in astigmatism and how modulated neural information causes a perceptual error. We induced astigmatism by placing a cylindrical lens on the dominant eye of human participants, while they reported the orientations of the presented Gabor patches. The simultaneously recorded electroencephalogram activity revealed that stimulus orientation information estimated from the multivariate electroencephalogram activity was biased away from the neural representation of the astigmatic axis and predictive of behavioral bias. The representational neural dynamics underlying the perceptual error revealed the temporal state transition; it was transiently dynamic and unstable (approximately 350 ms from stimulus onset) that soon stabilized. The biased stimulus orientation information represented by the spatially distributed electroencephalogram activity mediated the distorted retinal images and biased orientation perception in induced astigmatism.

Ensemble representations reveal distinct neural coding of visual working memory.

We characterized the population-level neural coding of ensemble representations in visual working memory from human electroencephalography. Ensemble representations provide a unique opportunity to investigate structured representations of working memory because the visual system encodes high-order summary statistics as well as noisy sensory inputs in a hierarchical manner. Here, we consistently observe stable coding of simple features as well as the ensemble mean in frontocentral electrodes, which even correlated with behavioral indices of the ensemble across individuals. In occipitoparietal electrodes, however, we find that remembered features are dynamically coded over time, whereas neural coding of the ensemble mean is absent in the old/new judgment task. In contrast, both dynamic and stable coding are found in the continuous estimation task. Our findings suggest that the prefrontal cortex holds behaviorally relevant abstract representations while visual representations in posterior and visual areas are modulated by the task demands.

Motion direction representation in multivariate electroencephalography activity for smooth pursuit eye movements.

Visually-guided smooth pursuit eye movements are composed of initial open-loop and later steady-state periods. Feedforward sensory information dominates the motor behavior during the open-loop pursuit, and a more complex feedback loop regulates the steady-state pursuit. To understand the neural representations of motion direction during open-loop and steady-state smooth pursuits, we recorded electroencephalography (EEG) responses from human observers while they tracked random-dot kinematograms as pursuit targets. We estimated population direction tuning curves from multivariate EEG activity using an inverted encoding model. We found significant direction tuning curves as early as about 60 ms from stimulus onset. Direction tuning responses were generalized to later times during the open-loop smooth pursuit, but they became more dynamic during the later steady-state pursuit. The encoding quality of retinal motion direction information estimated from the early direction tuning curves was predictive of trial-by-trial variation in initial pursuit directions. These results suggest that the movement directions of open-loop smooth pursuit are guided by the representation of the retinal motion present in the multivariate EEG activity.

Cortical sources of Vernier acuity in the human visual system: An EEG-source imaging study.

Vernier acuity determines the relative position of visual features with a precision better than the sampling resolution of cone receptors in the retina. Because Vernier displacement is thought to be mediated by orientation-tuned mechanisms, Vernier acuity is presumed to be processed in striate visual cortex (V1). However, there is considerable evidence suggesting that Vernier acuity is dependent not only on structures in V1 but also on processing in extrastriate cortical regions. Here we used functional magnetic resonance imaging-informed electroencephalogram source imaging to localize the cortical sources of Vernier acuity in observers with normal vision. We measured suprathreshold and near-threshold responses to Vernier onset/offset stimuli at different stages of the visual cortical hierarchy, including V1, hV4, lateral occipital cortex (LOC), and middle temporal cortex (hMT+). These responses were compared with responses to grating on/off stimuli, as well as to stimuli that control for lateral motion in the Vernier task. Our results show that all visual cortical regions of interest (ROIs) responded to both suprathreshold Vernier and grating stimuli. However, thresholds for Vernier displacement (Vernier acuity) were lowest in V1 and LOC compared with hV4 and hMT+, whereas all visual ROIs had identical thresholds for spatial frequency (grating acuity) and for relative motion. The cortical selectivity of sensitivity to Vernier displacement provides strong evidence that LOC, in addition to V1, is involved in Vernier acuity processing. The robust activation of LOC might be related to the sensitivity to the relative position of features, which is common to Vernier displacement and to some kinds of texture segmentation.

Attention to Multiple Objects Facilitates Their Integration in Prefrontal and Parietal Cortex.

Selective attention is known to interact with perceptual organization. In visual scenes, individual objects that are distinct and discriminable may occur on their own, or in groups such as a stack of books. The main objective of this study is to probe the neural interaction that occurs between individual objects when attention is directed toward one or more objects. Here we record steady-state visual evoked potentials via electrocorticography to directly assess the responses to individual stimuli and to their interaction. When human participants attend to two adjacent stimuli, prefrontal and parietal cortex shows a selective enhancement of only the neural interaction between stimuli, but not the responses to individual stimuli. When only one stimulus is attended, the neural response to that stimulus is selectively enhanced in prefrontal and parietal cortex. In contrast, early visual areas generally manifest responses to individual stimuli and to their interaction regardless of attentional task, although a subset of the responses is modulated similarly to prefrontal and parietal cortex. Thus, the neural representation of the visual scene as one progresses up the cortical hierarchy becomes more highly task-specific and represents either individual stimuli or their interaction, depending on the behavioral goal. Attention to multiple objects facilitates an integration of objects akin to perceptual grouping.SIGNIFICANCE STATEMENT Individual objects in a visual scene are seen as distinct entities or as parts of a whole. Here we examine how attention to multiple objects affects their neural representation. Previous studies measured single-cell or fMRI responses and obtained only aggregate measures that combined the activity to individual stimuli as well as their potential interaction. Here, we directly measure electrocorticographic steady-state responses corresponding to individual objects and to their interaction using a frequency-tagging technique. Attention to two stimuli increases the interaction component that is a hallmark for perceptual integration of stimuli. Furthermore, this stimulus-specific interaction is represented in prefrontal and parietal cortex in a task-dependent manner.

Degraded attentional modulation of cortical neural populations in strabismic amblyopia.

Behavioral studies have reported reduced spatial attention in amblyopia, a developmental disorder of spatial vision. However, the neural populations in the visual cortex linked with these behavioral spatial attention deficits have not been identified. Here, we use functional MRI-informed electroencephalography source imaging to measure the effect of attention on neural population activity in the visual cortex of human adult strabismic amblyopes who were stereoblind. We show that compared with controls, the modulatory effects of selective visual attention on the input from the amblyopic eye are substantially reduced in the primary visual cortex (V1) as well as in extrastriate visual areas hV4 and hMT+. Degraded attentional modulation is also found in the normal-acuity fellow eye in areas hV4 and hMT+ but not in V1. These results provide electrophysiological evidence that abnormal binocular input during a developmental critical period may impact cortical connections between the visual cortex and higher level cortices beyond the known amblyopic losses in V1 and V2, suggesting that a deficit of attentional modulation in the visual cortex is an important component of the functional impairment in amblyopia. Furthermore, we find that degraded attentional modulation in V1 is correlated with the magnitude of interocular suppression and the depth of amblyopia. These results support the view that the visual suppression often seen in strabismic amblyopia might be a form of attentional neglect of the visual input to the amblyopic eye.

The influence of segmentation and uncertainty on target selection.

This study examines how two factors affect target selection: the contiguity of the target with the surrounding surface and certainty about target location. Previous studies indicate that a target among distractors is easier to find when the search items are on the same surface rather than different surfaces. In contrast, our recent study indicates that when the target is in a known location, sensitivity to the target is higher when it is clearly separated from the surrounding surface. Here we examine the effects of both surface contiguity and uncertainty about target location on contrast discrimination. Observers were asked to detect a contrast change on a grating target that was either segmented or contiguous with the surround grating and occurred either at a known or unknown location. Thresholds for contrast discrimination depended critically on both segmentation and location uncertainty. When the contrast change appeared at a known location isolated from the background, segmentation aided the selection of the target location, but when the contrast change occurred at an unknown location on a contiguous background, grouping of the surface as a single entity aided the detection of the target location as a discontinuity from the surface.

Attention selects informative neural populations in human V1.

In a neural population driven by a simple grating stimulus, different subpopulations are maximally informative about changes to the grating's orientation and contrast. In theory, observers should attend to the optimal subpopulation when switching between orientation and contrast discrimination tasks. Here we used source-imaged, steady-state visual evoked potentials and visual psychophysics to determine whether this is the case. Observers fixated centrally while static targets were presented bilaterally along with a cue indicating task type (contrast or orientation modulation detection) and task location (left or right). Changes in neuronal activity were measured by quantifying frequency-tagged responses from flickering "reporter" gratings surrounding the targets. To determine the orientation tuning of attentionally modulated neurons, we measured responses for three different probe-reporter angles: 0, 20, and 45°. We estimated frequency-tagged cortical activity using a minimum norm inverse procedure combined with realistic MR-derived head models and retinotopically mapped visual areas. Estimates of neural activity from regions of interest centered on V1 showed that attention to a spatial location clearly increased the amplitude of the neural response in that location. More importantly, the pattern of modulation depended on the task. For orientation discrimination, attentional modulation showed a sharp peak in the population tuned 20° from the target orientation, whereas for contrast discrimination the enhancement was more broadly tuned. Similar tuning functions for orientation and contrast discrimination were obtained from psychophysical adaptation studies. These findings indicate that humans attend selectively to the most informative neural population and that these populations change depending on the nature of the task.

The selectivity of task-dependent attention varies with surrounding context.

Attention is thought to operate by enhancing the target of interest and suppressing the surroundings. We hypothesized that the spatial profile of attention depends on the surround's relationship to the target. Using high-density electroencephalographic measurements, we examined the spatial profile of attention to a grating target surrounded by an annular grating that was either coextensive with the target (unsegmented) or appeared segmented from it due to a gap or phase offset. We directly probed the spread of attention from the central target into the surround by flickering the surround and monitoring frequency-tagged steady-state visual-evoked potentials. Observers were required to detect a contrast increment that occurred only on the target. Successful detection of the increment required selecting the target and suppressing the surround, particularly when the target did not readily segment from the surround. The profile of attention was investigated in five visual regions of interest (ROIs) (V1, V4, V3A, lateral occipital complex, and human middle temporal area), mapped in a separate anatomical magnetic resonance imaging scan. We found that in most ROIs, attention to the target generated smaller responses from the surrounding annulus when it was contiguous compared with when it was clearly segmented. This result shows that the profile of attention depends on task demands and on surrounding context; attention is tightly focused when the target region needs to be isolated but loosely focused when the target region is clearly segmented.

Internal curvature signal and noise in low- and high-level vision.

How does internal processing contribute to visual pattern perception? By modeling visual search performance, we estimated internal signal and noise relevant to perception of curvature, a basic feature important for encoding of three-dimensional surfaces and objects. We used isolated, sparse, crowded, and face contexts to determine how internal curvature signal and noise depended on image crowding, lateral feature interactions, and level of pattern processing. Observers reported the curvature of a briefly flashed segment, which was presented alone (without lateral interaction) or among multiple straight segments (with lateral interaction). Each segment was presented with no context (engaging low-to-intermediate-level curvature processing), embedded within a face context as the mouth (engaging high-level face processing), or embedded within an inverted-scrambled-face context as a control for crowding. Using a simple, biologically plausible model of curvature perception, we estimated internal curvature signal and noise as the mean and standard deviation, respectively, of the Gaussian-distributed population activity of local curvature-tuned channels that best simulated behavioral curvature responses. Internal noise was increased by crowding but not by face context (irrespective of lateral interactions), suggesting prevention of noise accumulation in high-level pattern processing. In contrast, internal curvature signal was unaffected by crowding but modulated by lateral interactions. Lateral interactions (with straight segments) increased curvature signal when no contextual elements were added, but equivalent interactions reduced curvature signal when each segment was presented within a face. These opposing effects of lateral interactions are consistent with the phenomena of local-feature contrast in low-level processing and global-feature averaging in high-level processing.

Differential roles of frequency-following and frequency-doubling visual responses revealed by evoked neural harmonics.

Frequency-following and frequency-doubling neurons are ubiquitous in both striate and extrastriate visual areas. However, responses from these two types of neural populations have not been effectively compared in humans because previous EEG studies have not successfully dissociated responses from these populations. We devised a light-dark flicker stimulus that unambiguously distinguished these responses as reflected in the first and second harmonics in the steady-state visual evoked potentials. These harmonics revealed the spatial and functional segregation of frequency-following (the first harmonic) and frequency-doubling (the second harmonic) neural populations. Spatially, the first and second harmonics in steady-state visual evoked potentials exhibited divergent posterior scalp topographies for a broad range of EEG frequencies. The scalp maximum was medial for the first harmonic and contralateral for the second harmonic, a divergence not attributable to absolute response frequency. Functionally, voluntary visual-spatial attention strongly modulated the second harmonic but had negligible effects on the simultaneously elicited first harmonic. These dissociations suggest an intriguing possibility that frequency-following and frequency-doubling neural populations may contribute complementary functions to resolve the conflicting demands of attentional enhancement and signal fidelity--the frequency-doubling population may mediate substantial top-down signal modulation for attentional selection, whereas the frequency-following population may simultaneously preserve relatively undistorted sensory qualities regardless of the observer's cognitive state.

Attention induces synchronization-based response gain in steady-state visual evoked potentials.

When attention is voluntarily directed to a spatial location, visual sensitivity increases at that location. What causes this improved sensitivity? Studies of single neuron spike rates in monkeys have provided mixed results in regard to whether attending to a stimulus increases its effective contrast (contrast gain) or multiplicatively boosts stimulus-driven neural responses (response or activity gain). We monitored frequency-tagged steady-state visual evoked potentials (SSVEPs) in humans and found that voluntary sustained attention multiplicatively increased stimulus-driven population electrophysiological activity. Analyses of intertrial phase coherence showed that this attentional response gain was at least partially due to the increased synchronization of SSVEPs to stimulus flicker. These results suggest that attention operates in a complementary manner at different levels; attention seems to increase single-neuron spike rates in a variety of ways, including contrast, response and activity gains, while also inducing a multiplicative boost on neural population activity via enhanced response synchronization.

Stochastic resonance in binocular rivalry.

When a different image is presented to each eye, visual awareness spontaneously alternates between the two images--a phenomenon called binocular rivalry. Because binocular rivalry is characterized by two marginally stable perceptual states and spontaneous, apparently stochastic, switching between them, it has been speculated that switches in perceptual awareness reflect a double-well-potential type computational architecture coupled with noise. To characterize this noise-mediated mechanism, we investigated whether stimulus input, neural adaptation, and inhibitory modulations (thought to underlie perceptual switches) interacted with noise in such a way that the system produced stochastic resonance. By subjecting binocular rivalry to weak periodic contrast modulations spanning a range of frequencies, we demonstrated quantitative evidence of stochastic resonance in binocular rivalry. Our behavioral results combined with computational simulations provided insights into the nature of the internal noise (its magnitude, locus, and calibration) that is relevant to perceptual switching, as well as provided novel dynamic constraints on computational models designed to capture the neural mechanisms underlying perceptual switching.