Similar neural representations of the target for saccades and perception during search.

Are the body's actions and the mind's perceptions the result of shared neural processing, or are they performed largely independently? The brain has two major processing streams, and some have proposed that this division segregates visual processing for action and perception. The ventral pathway is claimed to support conscious experience (perception), whereas the dorsal pathway is claimed to support the control of movement (action). Others have argued that perception and action share much of their visual processing within the primate cortex. During visual search, the brain performs a sophisticated deployment of eye movements (saccadic actions) to gather information to subserve perceptual judgments. The relationship between the neural mechanisms mediating perception and action in visual search remains unexplored. Here, we investigate the visual representation of target information in the human brain, both for perceptual decisions and for saccadic actions during visual search. We use classification image analysis, a form of reverse correlation, to estimate the behavioral receptive fields of the visual mechanisms responsible for saccadic and perceptual responses during the same visual search task. Results show that the behavioral receptive fields mediating the perceptual decisions are indistinguishable from those driving the oculomotor decisions, suggesting that similar neural mechanisms are responsible for both perception and oculomotor action during search. Diverging target representations would result in an inefficient coupling between eye movement planning and perceptual judgments. Thus, a common target representation would be more optimal and might be expected to have evolved through natural selection in the neural systems responsible for visual search.

Limited flexibility in the filter underlying saccadic targeting.

The choice of where to look in a visual scene depends on visual processing of information from potential target locations. We examined to what extent the sampling window, or filter, underlying saccadic eye movements is under flexible control and adjusted to the behavioural task demands. Observers performed a contrast discrimination task with systematic variations in the spatial scale and location of the visual signals: small (sigma=0.175 degrees ) or large (sigma=0.8 degrees ) Gaussian signals were presented 4.5 degrees , 6 degrees , or 9 degrees away from central fixation. In experiment 1, we measured the accuracy of the first saccade as a function of target contrast. The efficiency of saccadic targeting decreased with increases in both scale and eccentricity. In experiment 2, the filter underlying saccadic targeting was estimated with the classification image method. We found that the filter (1) had a center-surround organisation, even though the signal was Gaussian; (2) was much too small for the large scale items; (3) remained constant up to the largest measured eccentricity of 9 degrees . The filter underlying the decision of where to look is not fixed, and can be adjusted to the task demands. However, there are clear limits to this flexibility. These limits reflect the coding of visual information by early mechanisms, and the extent to which the neural circuitry involved in programming saccadic eye movements is able to appropriately weigh and combine the outputs from these mechanisms.

The time course of visual information accrual guiding eye movement decisions.

Saccadic eye movements are the result of neural decisions about where to move the eyes. These decisions are based on visual information accumulated before the saccade; however, during an approximately 100-ms interval immediately before the initiation of an eye movement, new visual information cannot influence the decision. Does the brain simply ignore information presented during this brief interval or is the information used for the subsequent saccade? Our study examines how and when the brain integrates visual information through time to drive saccades during visual search. We introduce a new technique, saccade-contingent reverse correlation, that measures the time course of visual information accrual driving the first and second saccades. Observers searched for a contrast-defined target among distractors. Independent contrast noise was added to the target and distractors every 25 ms. Only noise presented in the time interval in which the brain accumulates information will influence the saccadic decisions. Therefore, we can retrieve the time course of saccadic information accrual by averaging the time course of the noise, aligned to saccade initiation, across all trials with saccades to distractors. Results show that before the first saccade, visual information is being accumulated simultaneously for the first and second saccades. Furthermore, information presented immediately before the first saccade is not used in making the first saccadic decision but instead is stored and used by the neural processes driving the second saccade.

Saccadic and perceptual performance in visual search tasks. I. Contrast detection and discrimination.

Humans use saccadic eye movements when they search for visual targets. We investigated the relationship between the visual processing used by saccades and perception during search by comparing saccadic and perceptual decisions under conditions in which each had access to equal visual information. We measured the accuracy of perceptual judgments and of the first search saccade over a wide range of target saliences [signal-to-noise ratios (SNRs)] in both a contrast-detection and a contrast-discrimination task. We found that saccadic and perceptual performances (1) were similar across SNRs, (2) showed similar task-dependent differences, and (3) were well described by a model based on signal detection theory that explicitly includes observer uncertainty [M. P. Eckstein et al., J. Opt. Soc. Am. A 14, 2406 (1997)1]. Our results demonstrate that the accuracy of the first saccade provides much information about the observer's perceptual state at the time of the saccadic decision and provide evidence that saccades and perception use similar visual processing mechanisms for contrast detection and discrimination.

Saccadic and perceptual performance in visual search tasks. II. Letter discrimination.

Can the oculomotor system use shape cues to guide search saccades? Observers searched for target letters (D, U, or X) among distractors (the letter O in the discrimination task and blank locations in the detection task) in Gaussian white noise. We measured the accuracy of first saccadic responses on each trial and perceptual (i.e., button-press) responses in separate trials with the stimulus duration chosen so that the saccadic and perceptual processing times were matched. We calculated the relative efficiency of saccadic decisions compared with perceptual decisions, eta(rel) = (d'(sac)/d'(per))2. Relative efficiency was low but consistently greater than zero in discrimination tasks (15% +/- 6%) and high in detection tasks (60% +/- 10%). We conclude that the saccadic targeting system can use shape cues, but less efficiently than the perceptual system can.

Human discrimination of visual direction of motion with and without smooth pursuit eye movements.

It has long been known that ocular pursuit of a moving target has a major influence on its perceived speed (Aubert, 1886; Fleischl, 1882). However, little is known about the effect of smooth pursuit on the perception of target direction. Here we compare the precision of human visual-direction judgments under two oculomotor conditions (pursuit vs. fixation). We also examine the impact of stimulus duration (200 ms vs. ~800 ms) and absolute direction (cardinal vs. oblique). Our main finding is that direction discrimination thresholds in the fixation and pursuit conditions are indistinguishable. Furthermore, the two oculomotor conditions showed oblique effects of similar magnitudes. These data suggest that the neural direction signals supporting perception are the same with or without pursuit, despite remarkably different retinal stimulation. During fixation, the stimulus information is restricted to large, purely peripheral retinal motion, while during steady-state pursuit, the stimulus information consists of small, unreliable foveal retinal motion and a large efference-copy signal. A parsimonious explanation of our findings is that the signal limiting the precision of direction judgments is a neural estimate of target motion in head-centered (or world-centered) coordinates (i.e., a combined retinal and eye motion signal) as found in the medial superior temporal area (MST), and not simply an estimate of retinal motion as found in the middle temporal area (MT).