Human short-latency ocular vergence responses produced by interocular velocity differences.

We studied human short-latency vergence eye movements to a novel stimulus that produces interocular velocity differences without a changing disparity signal. Sinusoidal luminance gratings moved in opposite directions (left vs. right; up vs. down) in the two eyes. The grating seen by each eye underwent »-wavelength shifts with each image update. This arrangement eliminated changing disparity cues, since the phase difference between the eyes alternated between 0° and 180°. We nevertheless observed robust short-latency vergence responses (VRs), whose sign was consistent with the interocular velocity differences (IOVDs), indicating that the IOVD cue in isolation can evoke short-latency VRs. The IOVD cue was effective only when the images seen by the two eyes overlapped in space. We observed equally robust VRs for opposite horizontal motions (left in one eye, right in the other) and opposite vertical motions (up in one eye, down in the other). Whereas the former are naturally generated by objects moving in depth, the latter are not part of our normal experience. To our knowledge, this is the first demonstration of a behavioral consequence of vertical IOVD. This may reflect the fact that some neurons in area MT are sensitive to these motion signals (Czuba, Huk, Cormack, & Kohn, 2014). VRs were the strongest for spatial frequencies in the range of 0.35-1 c/°, much higher than the optimal spatial frequencies for evoking ocular-following responses observed during frontoparallel motion. This suggests that the two motion signals are detected by different neuronal populations. We also produced IOVD using moving uncorrelated one-dimensional white-noise stimuli. In this case the most effective stimuli have low speed, as predicted if the drive originates in neurons tuned to high spatial frequencies (Sheliga, Quaia, FitzGibbon, & Cumming, 2016).

Anisotropy in spatial summation properties of human Ocular-Following Response (OFR).

Using sinusoidal gratings we show that an increase in stimulus size confined to the dimension orthogonal to the axis of motion leads to stronger Ocular Following Responses (OFRs) up to a certain optimal size. An increase beyond this optimum produces smaller responses, indicating suppressive interactions. In sharp contrast, when the stimulus growth occurs parallel to the axis of motion OFR magnitudes increase monotonically both for horizontal and vertical directions of motion. Similar results are obtained with 1D white noise patterns. However, the OFR spatial anisotropy is minimal with 2D white noise patterns, revealing a pivotal role of orientation-selective (i.e., cortical) mechanisms in mediating this phenomenon. The lack of anisotropy for 2D patterns suggests that directional signals alone are not sufficient to elicit this suppression. The OFR spatial anisotropy is potentiated if a stationary grating is presented for 600-1000ms before its motion commences, further emphasizing the importance of static orientation signals. These results suggest that the strength of cortical spatial interactions is asymmetric-i.e., larger in the direction of the ends than the flanks of an orientation-selective receptive field-which corroborates the existing neurophysiological evidence.

Retinal visual processing constrains human ocular following response.

Ocular following responses (OFRs) are the initial tracking eye movements elicited at ultra-short latency by sudden motion of a textured pattern. We wished to evaluate quantitatively the impact that subcortical stages of visual processing might have on the OFRs. In three experiments we recorded the OFRs of human subjects to brief horizontal motion of 1D vertical sine-wave gratings restricted to an elongated horizontal aperture. Gratings were composed of a variable number of abutting horizontal strips where alternate strips were in counterphase. In one of the experiments we also utilized gratings occupying a variable number of horizontal strips separated vertically by mean-luminance gaps. We modeled retinal center/surround receptive fields as a difference of two 2-D Gaussian functions. When the characteristics of such local filters were selected in accord with the known properties of primate retinal ganglion cells, a single-layer model was capable to quantitatively account for the observed changes in the OFR amplitude for stimuli composed of counterphase strips of different heights (Experiment 1), for a wide range of stimulus contrasts (Experiment 2) and spatial frequencies (Experiment 3). A similar model using oriented filters that resemble cortical simple cells was also able to account for these data. Since similar filters can be constructed from the linear summation of retinal filters, and these filters alone can explain the data, we conclude that retinal processing determines the response to these stimuli. Thus, with appropriately chosen stimuli, OFRs can be used to study visual spatial integration processes as early as in the retina.

Spatial summation properties of the human ocular following response (OFR): dependence upon the spatial frequency of the stimulus.

Ocular following responses (OFRs) are the initial tracking eye movements that can be elicited at ultra-short latency by sudden motion of a textured pattern. The OFR magnitude depends upon stimulus size, and also upon the spatial frequency (SF) of sine-wave gratings. Here we investigate the interaction of size and SF. We recorded initial OFRs in human subjects when 1D vertical sine-wave gratings were subject to horizontal motion. Gratings were restricted to elongated horizontal apertures-"strips"-aligned with the axis of motion. In Experiment 1 the SF and the height of a single strip was manipulated. The magnitude of the OFR increased with strip height up to some optimum value, while strip heights greater than this optimum produced smaller responses. This effect was strongly dependent on SF: the optimum strip height was smaller for higher SFs. In order to explore the underlying mechanism, Experiment 2 measured OFRs to stimuli composed of two thin horizontal strips-one in the upper visual field, the other in the lower visual field-whose vertical separation varied 32-fold. Stimuli of different sizes can be reconstructed from the sum of such horizontal strips. We found that the OFRs in Experiment 1 were smaller than the sum of the responses to the component stimuli, but greater than the average of those responses. We defined an averaging coefficient that described whether a given response was closer to the sum or to the average. For any one SF, the averaging coefficients were similar over a wide range of stimulus sizes, while they varied considerably (7-fold) for stimuli of different SFs.

A quantitative explanation of responses to disparity-defined edges in macaque V2.

Previous experiments have shown that V2 neurons respond to complex stimuli such as cyclopean edges (edges defined purely by binocular disparity), angles, and motion borders. It is currently unknown whether these responses are a simple consequence of converging inputs from a prior stage of processing (V1). Alternatively, they may identify edges in a way that is invariant across a range of visual cues defining the edge, in which case they could provide a neuronal substrate for scene segmentation. Here, we examine the ability of a simple feedforward model that combines two V1-like inputs to describe the responses of V2 neurons to cyclopean edges. A linear feedforward model was able to qualitatively reproduce the major patterns of response enhancement for cyclopean edges seen in V2. However, quantitative fitting revealed that this model usually predicts response suppression by some edge configurations and such suppression was rarely seen in the data. This problem was resolved by introducing a squaring nonlinearity at the output of the individual inputs prior to combination. The extended model produced extremely good fits to most of our data. We conclude that the responses of V2 neurons to complex stimuli such as cyclopean edges can be adequately explained by a simple convergence model and do not necessarily represent the development of sophisticated mechanisms that signal scene segmentation, although they probably constitute a step toward this goal.

Comparing perceptual signals of single V5/MT neurons in two binocular depth tasks.

Neurons in the extrastriate visual area V5/MT show perceptually relevant signals in binocular depth tasks, which can be measured as a choice probability (CP) for the neuron. The presence of a CP in a particular paradigm may be an indicator that the neuron is generally part of the substrate for the perception of binocular depth. We compared the responses of those single neurons that show CPs in one stereoscopic depth task with their responses in another stereo task. Each neuron was tested for the presence of 1) CPs during a task in which macaques responded to the sign of binocular depth in a structure-from-motion stimulus, to judge its direction of three-dimensional rotation and 2) a consistent response to the stereo disparity of binocularly anti-correlated stimuli. Previous work, confirmed here, shows that changing the disparity of these binocularly anti-correlated stimuli often fails to yield a coherent change in the depth percept. For each test alone, there are V5/MT neurons that carry signals that are congruent with the perceptual effects. However, on comparing tests, there is no fixed pool of neurons that can account for the binocular depth percept. Excitation of neurons with a measurable CP does not necessarily lead to a change in perception. The cortical circuitry must be able to make dynamic changes in the pools of neurons that underlie perceptual judgments according to the demands of the task.

An unexpected specialization for horizontal disparity in primate primary visual cortex.

The horizontal separation of the eyes means that objects nearer or farther than the fixation point project to different locations on the two retinae, differing principally in their horizontal coordinates (horizontal binocular disparity). Disparity-selective neurons have generally been studied with disparities applied in only one direction (often horizontal), which cannot determine whether the encoding is specialized for processing disparities along the horizontal axis. It is therefore unclear if disparity selectivity represents a specialization for naturally occurring disparities. I used random dot stereograms to study disparity-selective neurons from the primary visual cortex (V1) of awake fixating monkeys. Many combinations of vertical and horizontal disparity were used, characterizing the surface of responses as a function of two-dimensional disparity. Here I report that the response surface usually showed elongation along the horizontal disparity axis, despite the isotropic stimulus. Thus these neurons modulated their firing rate over a wider range of horizontal disparity than vertical disparity. This demonstrates that disparity-selective cells are specialized for processing horizontal disparity, and that existing models of disparity selectivity require substantial revision.

A specialization for relative disparity in V2.

Stereoscopic depth perception relies on binocular disparities, or small geometric differences between the retinal images of each eye. The most reliable binocular depth judgments are those that are based on relative disparities between two simultaneously visible features in a scene. Many cortical areas contain neurons that are sensitive to disparity, but it is unclear whether any areas show a specific sensitivity to relative disparity. We recorded from neurons in the early cortical visual area V2 of the awake macaque during presentation of random-dot patterns. The depth of a central region ('center'), and that of an annular surrounding region ('surround'), were manipulated independently in these stimuli. Some cells were fully selective for the resulting relative disparities. Most showed partial selectivity, which nonetheless indicated a sensitivity for the depth relationship between center and surround. Both types of neural response could support psychophysical judgments of relative depth.

Quantitative analysis of the responses of V1 neurons to horizontal disparity in dynamic random-dot stereograms.

Horizontal disparity tuning for dynamic random-dot stereograms was investigated for a large population of neurons (n = 787) in V1 of the awake macaque. Disparity sensitivity was quantified using a measure of the discriminability of the maximum and minimum points on the disparity tuning curve. This measure and others revealed a continuum of selectivity rather than separate populations of disparity- and nondisparity-sensitive neurons. Although disparity sensitivity was correlated with the degree of direction tuning, it was not correlated with other significant neuronal properties, including preferred orientation and ocular dominance. In accordance with the Gabor energy model, tuning curves for horizontal disparity were adequately described by Gabor functions when the neuron's orientation preference was near vertical. For neurons with orientation preferences near to horizontal, a Gaussian function was more frequently sufficient. The spatial frequency of the Gabor function that described the disparity tuning was weakly correlated with measurements of the spatial frequency and orientation preference of the neuron for drifting sinusoidal gratings. Energy models make several predictions about the relationship between the response rates to monocular and binocular dot patterns. Few of the predictions were fulfilled exactly, although the observations can be reconciled with the energy model by simple modifications. These same modifications also provide an account of the observed continuum in strength of disparity selectivity. A weak correlation between the disparity sensitivity of simultaneously recorded single- and multiunit data were revealed as well as a weak tendency to show similar disparity preferences. This is compatible with a degree of local clustering for disparity sensitivity in V1, although this is much weaker than that reported in area MT.

Range and mechanism of encoding of horizontal disparity in macaque V1.

The responses of single cortical neurons were measured as a function of the binocular disparity of dynamic random dot stereograms for a large sample of neurons (n = 787) from V1 of the awake macaque. From this sample, we selected 180 neurons whose tuning curves were strongly tuned for disparity, well sampled and well described by one-dimensional Gabor functions. The fitted parameters of the Gabor functions were used to resolve three outstanding issues in binocular stereopsis. First, we considered whether tuning curves can be meaningfully divided into discrete tuning types. Careful examination of the distributions of the Gabor parameters that determine tuning shape revealed no evidence for clustering. We conclude that a continuum of tuning types is present. Second, we investigated the mechanism of disparity encoding for V1 neurons. The shape of the disparity tuning function can be used to distinguish between position-encoding (in which disparity is encoded by an interocular shift in receptive field position) and phase-encoding (in which disparity is encoded by a difference in the receptive field profile in the 2 eyes). Both position and phase encoding were found to be common. This was confirmed by an independent assessment of disparity encoding based on the measurement of disparity sensitivity for sinusoidal luminance gratings of different spatial frequencies. The contributions of phase and position to disparity encoding were compared by estimating a population average of the rate of change in firing rate per degree of disparity. When this was calculated separately for the phase and position contributions, they were found to be closely similar. Third, we investigated the range of disparity tuning in V1 as a function of eccentricity in the parafoveal range. We find few cells which are selective for disparities greater than +/-1 degrees even at the largest eccentricity of approximately 5 degrees. The preferred disparity was correlated with the spatial scale of the tuning curve, and for most units lay within a +/-pi radians phase limit. Such a size-disparity correlation is potentially useful for the solution of the correspondence problem.

Cortical mechanisms of binocular stereoscopic vision.

The early neurophysiology of binocular vision is largely dominated by measurements of disparity selectivity in cortical neurons in various visual areas. Incisive progress has been made by the intensive study of the mechanism of disparity selectivity of V1 in cortical neurons and the development of a number of tests for the involvement of single neurons in the perception of stereoscopic depth. The picture that now emerges is that cortical area V1 must be a preliminary processing stage for the analysis of stereoscopic depth, whereas some of the extrastriate areas may actually be responsible for the generation of neuronal signals that underlie the perception of binocular depth.

Responses of macaque V1 neurons to binocular orientation differences.

Interocular differences in orientation occur during binocular viewing of a surface slanted in depth. These orientation disparities could be exploited by the visual system to provide information about surface slant, but gradients of positional disparity provide an equally effective means to the same end. We examined the encoding of orientation disparities in V1 neurons that were recorded from two awake fixating monkeys. Monocular orientation selectivity was measured separately in each eye. Although the preferred monocular orientation in the left and right eyes was highly correlated (r = 0.98), 19 of 61 cells showed a significant interocular difference in preferred orientation (IDPO). By itself, an IDPO does not imply a specific binocular selectivity for orientation differences. We therefore examined the response to 25 binocular combinations of orientations by pairing each of five orientations in one eye with five in the other. Forty-four of 64 neurons showed responses that reflected the monocular orientation tuning selectivity; the preferred orientation disparity changed when the monocular orientation was changed in either eye. The remaining third (20 of 64) responded to a consistent orientation disparity in a way that was not simply predictable from monocular orientation selectivity. However, nearly all of these neurons were selective for positional disparity, and several characteristics of the responses suggest that the apparent selectivity for orientation disparities was just a consequence of the positional disparity sensitivity. Neither the data presented here nor previous data from the cat (Blakemore et al., 1972; Nelson et al., 1977) support the idea that a population of neurons early in the visual system has a separate encoding scheme for orientation disparities.

Perceptually bistable three-dimensional figures evoke high choice probabilities in cortical area MT.

The role of the primate middle temporal area (MT) in depth perception was examined by considering the trial-to-trial correlations between neuronal activity and reported depth sensations. A set of moving random dots portrayed a cylinder rotating about its principal axis. In this structure-from-motion stimulus, the direction of rotation is ambiguous and the resulting percept undergoes spontaneous fluctuations. The stimulus can be rendered unambiguous by the addition of binocular disparities. We trained monkeys to report the direction of rotation in a set of these stimuli, one of which had zero disparity. Many disparity-selective neurons in area MT are selective for the direction of rotation defined by disparity. Across repeated presentations of the ambiguous (zero-disparity) stimulus, there was a correlation between neuronal firing and the reported direction of rotation, as found by Bradley et al. (1998). Quantification of this effect using choice probabilities (Britten et al., 1996) allowed us to demonstrate that the correlation cannot be explained by eye movements, behavioral biases, or attention to spatial location. MT neurons therefore appear to be involved in the perceptual decision process. The mean choice probability (0.67) was substantially larger than that reported for MT neurons in a direction discrimination task (Britten et al., 1996). This implies that MT neurons make a different contribution to the two tasks. For the depth task, either the pool of neurons used is smaller or the correlation between neurons in the pool is larger.

The integration of orientation information in the motion correspondence problem.

We examine how differently oriented components contribute to the discrimination of motion direction along a horizontal axis. Stimuli were two-frame random-dot kinematograms that were narrowband filtered in spatial frequency. On each trial, subjects had to state whether motion was to the left or the right. For each stimulus condition, Dmax (the largest displacement supporting 80% correct direction discrimination performance) was measured. In experiment 1, Dmax was measured for orientationally narrowband stimuli as a function of their mean orientation. Dmax was found to increase as the orientation of the stimuli became closer to the axis of motion. Experiment 2 used isotropic stimuli in which some orientation bands contained a coherent motion signal, and some contained only noise. When the noise band started at vertical orientations and increased until only horizontal orientations contained a coherent motion signal, Dmax increased slightly. This suggests that near-vertical orientations interfere with motion perception at large displacements when they contain a coherent motion signal. When the noise band started at horizontal and increased until only vertical orientations contained the motion signal, Dmax decreased steadily. This implies that Dmax depends at least partly on the most horizontal motion signal in the stimulus. These results were contrasted with two models. In the first, the visual system utilises the most informative orientations (nearest horizontal). In the second, all available orientations are used equally. Results supported an intermediate interpretation, in which all orientations are used but more informative ones are weighted more heavily.

The physiology of stereopsis.

Binocular disparity provides the visual system with information concerning the three-dimensional layout of the environment. Recent physiological studies in the primary visual cortex provide a successful account of the mechanisms by which single neurons are able to signal disparity. This work also reveals that additional processing is required to make explicit the types of signal required for depth perception (such as the ability to match features correctly between the two monocular images). Some of these signals, such as those encoding relative disparity, are found in extrastriate cortex. Several other lines of evidence also suggest that the link between perception and neuronal activity is stronger in extrastriate cortex (especially MT) than in the primary visual cortex.

Modeling V1 neuronal responses to orientation disparity.

The contribution of interocular orientation differences to depth perception, at either the neuronal or the psychophysical level, is unclear. To understand the responses of binocular neurons to orientation disparity, we extended the energy model of Ohzawa et al. (1990) to incorporate binocular differences in receptive-field orientation. The responses of the model to grating stimuli with interocular orientation differences were examined, along with the responses to random dot stereograms (RDS) depicting slanted surfaces. The responses to combinations of stimulus orientations in the two eyes were left-right separable, which means there was no consistent response to the binocular orientation difference. All existing neuronal data concerning orientation disparity can be well described by this type of model (even a version with no disparity selectivity). The disparity sensitive model is nonetheless sensitive to changes in RDS slant, although it requires narrow orientation bandwidth to produce substantial modulation. The disparity-insensitive model shows no selectivity to slant in this stimulus. Several modifications to the model were attempted to improve its selectivity for orientation disparity and/or slant. A model built by summing several disparity-sensitive models showed left-right inseparable responses, responding maximally to a consistent orientation difference. Despite this property, the selectivity for slant in RDS stimuli was no better than the simple disparity-selective model. The range of models evaluated here demonstrate that interocular orientation differences are neither necessary nor sufficient for signaling slant. In contrast, within the framework of the energy model, positional disparity sensitivity appears to be both necessary and sufficient.

Local disparity not perceived depth is signaled by binocular neurons in cortical area V1 of the Macaque.

Binocular neurons that are closely related to depth perception should respond selectively for stimuli eliciting an appropriate depth sensation. To separate perceived depth from local disparity within the receptive field, sinusoidal luminance gratings were presented within a circular aperture. The disparity of the aperture was coupled to that of the grating, thereby rendering unambiguous the psychophysical matching between repeating cycles of the grating. In cases in which the stimulus disparity differs by one horizontal period of the grating, the portion of the grating that locally covers a receptive field is binocularly identical, but the depth sensation is very different because of the aperture. For 117 disparity-selective V1 neurons tested in two monkeys, the overwhelming majority responded equally well to configurations that were locally identical but led to different perceptions of depth. Because the psychophysical sensation is not reflected in the firing rate of V1 neurons, the signals that make stereo matches explicit are most likely elaborated in extrastriate cortex.

The precision of single neuron responses in cortical area V1 during stereoscopic depth judgments.

The performance of single neurons in cortical area V1 of alert macaque monkeys was compared against the animals' psychophysical performance during a binocular disparity discrimination task. Performance was assessed with stimuli that consisted of a patch of dynamic random dots, whose disparity varied from trial to trial, surrounded by an annulus of similar dots at a fixed disparity. On each trial, the animals indicated whether the depth of the central patch was in front of or behind the annulus. For each disparity of the center patch, neural performance was assessed by calculating the probability that the response of the neuron was greater or less than the response when the center disparity was the same as that of the annulus. Initially the animals performed the task simultaneously with the neural recording. However, the range of disparities used, which was appropriate for the neuronal recording, may have affected performance, because the thresholds were substantially lower (2.6x) when the psychophysical measurements were repeated later. Average neuronal thresholds were approximately 4x poorer than these behavioral thresholds, although the best neurons were marginally better than the animals' behavior. Thus, the well known precision of relative depth judgments can be supported with signals from a small number of V1 neurons. Interference with the relative depth information in the stimulus profoundly affected behavioral thresholds, which were approximately 10x poorer when the surround was absent or contained binocularly uncorrelated dots. In this case, single V1 neurons consistently outperform the observer: presumably here, psychophysical thresholds are limited by other factors (such as uncertainty about vergence eye position).

Binocular neurons in V1 of awake monkeys are selective for absolute, not relative, disparity.

Most neurophysiological accounts of disparity selectivity in neurons of the primary visual cortex (V1) imply that they are selective for absolute retinal disparities. By contrast, a number of psychophysical observations indicate that relative disparities play a more important role in depth perception. During recordings from disparity selective neurons in area V1 of awake behaving monkeys, we used a disparity feedback loop () to add controlled amounts of absolute disparity to a display containing both absolute and relative disparities. This manipulation changed the absolute disparity of all the visible features in the display but left unchanged the relative disparities signalled by these features. The addition of absolute disparities produced clear changes in the neural responses to unchanged external stimuli, which were well predicted by the measured change in absolute disparity: in 45/53 cases, the neuron maintained a consistent firing pattern with respect to absolute disparity so that the manipulation created no significant change in the absolute disparity preferred by the neuron. No neuron in V1 maintained a consistent relationship with relative disparity. We conclude that the relative disparity signals used in primate depth perception are constructed outside area V1.

Cortical area MT and the perception of stereoscopic depth.

Stereopsis is the perception of depth based on small positional differences between images formed on the two retinae (known as binocular disparity). Neurons that respond selectively to binocular disparity were first described three decades ago, and have since been observed in many visual areas of the primate brain, including V1, V2, V3, MT and MST. Although disparity-selective neurons are thought to form the neural substrate for stereopsis, the mere existence of disparity-selective neurons does not guarantee that they contribute to stereoscopic depth perception. Some disparity-selective neurons may play other roles, such as guiding vergence eye movements. Thus, the roles of different visual areas in stereopsis remain poorly defined. Here we show that visual area MT is important in stereoscopic vision: electrical stimulation of clusters of disparity-selective MT neurons can bias perceptual judgements of depth, and the bias is predictable from the disparity preference of neurons at the stimulation site. These results show that behaviourally relevant signals concerning stereoscopic depth are present in MT.