The integration of orientation information in the motion correspondence problem.

We examine how differently oriented components contribute to the discrimination of motion direction along a horizontal axis. Stimuli were two-frame random-dot kinematograms that were narrowband filtered in spatial frequency. On each trial, subjects had to state whether motion was to the left or the right. For each stimulus condition, Dmax (the largest displacement supporting 80% correct direction discrimination performance) was measured. In experiment 1, Dmax was measured for orientationally narrowband stimuli as a function of their mean orientation. Dmax was found to increase as the orientation of the stimuli became closer to the axis of motion. Experiment 2 used isotropic stimuli in which some orientation bands contained a coherent motion signal, and some contained only noise. When the noise band started at vertical orientations and increased until only horizontal orientations contained a coherent motion signal, Dmax increased slightly. This suggests that near-vertical orientations interfere with motion perception at large displacements when they contain a coherent motion signal. When the noise band started at horizontal and increased until only vertical orientations contained the motion signal, Dmax decreased steadily. This implies that Dmax depends at least partly on the most horizontal motion signal in the stimulus. These results were contrasted with two models. In the first, the visual system utilises the most informative orientations (nearest horizontal). In the second, all available orientations are used equally. Results supported an intermediate interpretation, in which all orientations are used but more informative ones are weighted more heavily.

Reversed stereo depth and motion direction with anti-correlated stimuli.

We used anti-correlated stimuli to compare the correspondence problem in stereo and motion. Subjects performed a two-interval forced-choice disparity/motion direction discrimination task for different displacements. For anti-correlated 1d band-pass noise, we found weak reversed depth and motion. With 2d anti-correlated stimuli, stereo performance was impaired, but the perception of reversed motion was enhanced. We can explain the main features of our data in terms of channels tuned to different spatial frequencies and orientation. We suggest that a key difference between the solution of the correspondence problem by the motion and stereo systems concerns the integration of information at different orientations.

Cue combination in the motion correspondence problem.

Image motion is a primary source of visual information about the world. However, before this information can be used the visual system must determine the spatio-temporal displacements of the features in the dynamic retinal image, which originate from objects moving in space. This is known as the motion correspondence problem. We investigated whether cross-cue matching constraints contribute to the solution of this problem, which would be consistent with physiological reports that many directionally selective cells in the visual cortex also respond to additional visual cues. We measured the maximum displacement limit (Dmax) for two-frame apparent motion sequences. Dmax increases as the number of elements in such sequences decreases. However, in our displays the total number of elements was kept constant while the number of a subset of elements, defined by a difference in contrast polarity, binocular disparity or colour, was varied. Dmax increased as the number of elements distinguished by a particular cue was decreased. Dmax was affected by contrast polarity for all observers, but only some observers were influenced by binocular disparity and others by colour information. These results demonstrate that the human visual system exploits local, cross-cue matching constraints in the solution of the motion correspondence problem.

The role of perspective effects and accelerations in perceived three-dimensional structure-from-motion.

It has been suggested that perceived three-dimensional (3D) structure-from-motion can be accounted for by a 2-frame orthographic approximation of the flow field. This study investigated the extent to which higher order cues (perspective and acceleration) are used in addition to first-order flow. Participants matched the 3D dihedral angle of a hinged plane (probe) defined by multiple-depth cues to one defined by motion only, for stimulus sizes of 8 and 33 degrees, using perspective and orthographic projection. The results show that perspective effects can be important even for relatively small stimuli (8 degrees) and that accelerations contribute to perceived shape. In all conditions, large biases were found. These are well accounted for by a model in which all relevant flow measurements (first-order, perspective, and acceleration) are used together with estimates of the noise in each. The model has no built-in bias toward particular 3D shapes. Instead, the visual system may act as an optimal estimator of 3D structure-from-motion.

Spatial frequency tuning for 3-D corrugations from motion parallax.

We provide evidence for the existence of multiple channels tuned to the spatial frequency of depth modulations defined by motion parallax. By linking the distortion of a random dot pattern to the horizontal position of an observer's head horizontally oriented 3-D corrugations were simulated in which the depth function consisted of a range of frequencies. In a baseline experiment thresholds were obtained for detecting depth modulations of single sinewaves for a range of spatial frequencies. In a masking experiment threshold signal strength was determined for detecting a signal frequency in the presence of noise with frequencies restricted to two bands around the signal component ('notched noise'). Threshold elevation was found to decrease with an increase in the spectral difference between signal and noise. By determining thresholds at various noise levels it was further established that the channel responded linearly in the tested range. Estimates of the bandwidth for spatial frequencies of 0.33 and 0.87 cycles/deg were both found to be 1.4 octaves. The results show that motion parallax processing is mediated by a series of narrowly tuned channels with bandwidths similar to those found for processing depth modulations defined by binocular disparity.

Weighted directional energy model of human stereo correspondence.

Previous work [Prince, S. J. D, & Eagle, R. A. (1999). Size-disparity correlation in human binocular depth perception. Proceedings of the Royal Society: Biological Sciences, 266, 1361-1365] has demonstrated that disparity sign discrimination performance in isolated bandpass patterns is supported at disparities much larger than a phase disparity model might predict. One possibility is that this extended performance relies on a separate second-order system [Hess, R. F., & Wilcox, L. M. (1994). Linear and non-linear filtering in stereopsis. Vision Research, 34, 2431-2438]. Here, a 'weighted directional energy' model is developed which explains a large body of crossed versus uncrossed disparity discrimination data with a single mechanism. This model assumes a population of binocular complex cells at every image point with a range of position disparity shifts. These cells sample a local energy function which is weighted so that energy at large disparities is relatively attenuated. Disparity sign is determined by summing and comparing energy at crossed and uncrossed disparities in the presence of noise. The model qualitatively predicts matching data for one-dimensional Gabor stimuli. This scheme also predicts DMax in Gabor stimuli and filtered noise. Moreover, a range of 'non-linear' phenomena, in which disparity is perceived from contrast envelope information alone, can be explained. The weighted directional energy model presents a biologically plausible, parsimonious explanation of matching behaviour in bandpass stimuli for both 'first-order' and 'second-order' stimuli which obviates the need for multiple mechanisms in stereo correspondence.

Stereo correspondence in one-dimensional Gabor stimuli.

Previous data [Prince, S.J.D., & Eagle, R.A., (1999). Size-disparity correlation in human binocular depth perception. Proceedings of the Royal Society of London B, 266, 1361-1365] have demonstrated that the upper disparity limit for stereopsis (DMax) is considerably smaller in filtered noise stereograms than in isolated Gabor patches of the same spatial frequency. This discrepancy is not currently understood. Here, the solution of the correspondence problem for bandpass stereograms was further examined. On each trial observers were presented with two one-dimensional Gabor stimuli containing disparities of equal magnitude but opposite sign. Subjects were required to indicate which interval contained the crossed disparity stimulus. It was found that matching behaviour changed as a function of Gabor envelope size. As a function of disparity magnitude, performance cycled between mostly correct and mostly incorrect at large envelope sizes but was always correct at small envelope sizes. At intermediate envelope sizes performance was cyclical at small disparities but always correct at large disparities. The critical envelope size at which performance changed from mostly correct to mostly incorrect at 270 degrees phase disparity was used as a measure of the matching performance as other parameters of the Gabor were varied. Both absolute and relative contrast were shown to influence the perceived sign of matches. Critical envelope size was also found to decrease as a function of spatial frequency, but more slowly than a phase-based limit would predict. These data cannot be predicted by current models of stereopsis, and can be used to constrain future models.

Size-disparity correlation in human binocular depth perception.

To use the small horizontal disparities between images projected to the eyes for the recovery of three-dimensional information, our visual system must first identify which feature in one eye's image corresponds with which in the other. The earliest level of disparity processing in primates (V1) contains cells that are spatial-frequency tuned. If such cells have a disparity range that covers only a single period of their mean tuning frequency, there will always be exactly one potential match within this range. Here, this 'size-disparity' hypothesis was tested by measuring the contrast sensitivity of stereopsis as a function of disparity for single bandpass-filtered items. It was found that thresholds were low and relatively constant up to disparities an order of magnitude larger than is predicted by this constraint. Furthermore, peak sensitivity was relatively independent of spatial frequency. A control experiment showed that binocular correlation of the carrier is necessary for this task. In a third experiment, the maximum disparity that supports threshold performance was compared for an isolated bandpass item and bandpass-filtered noise. This limit was found to be five times larger for the isolated stimuli. In summary, these findings show that the initial stage of disparity detection is not limited by the size-disparity constraint. For stimuli with multiple false targets, however, processes subsequent to this stage reduce the disparity range over which the correspondence problem can be solved.

The role of perspective information in the recovery of 3D structure-from-motion.

When investigating the recovery of three-dimensional structure-from-motion (SFM), vision scientists often assume that scaled-orthographic projection, which removes effects due to depth variations across the object, is an adequate approximation to full perspective projection. This is so even though SFM judgements can, in principle, be improved by exploiting perspective projection of scenes on to the retina. In an experiment, pairs of rotating hinged planes (open books) were simulated on a computer monitor, under either perspective or orthographic projection, and human observers were asked to indicate which they perceived had the larger dihedral angle. For small displays (4.6 x 6.0 degrees) discrimination thresholds were found to be similar under the two conditions, but diverged for all larger stimuli. In particular, as stimulus size was increased, performance under orthographic projection declined and by a stimulus size of 32 x 41 degrees performance was at chance for all subjects. In contrast, thresholds decreased under perspective projection as stimulus size was increased. These results show that human observers can use the information gained from perspective projection to recover SFM and that scaled-orthographic projection becomes an unacceptable approximation even at quite modest stimulus sizes. A model of SFM that incorporates measurement errors on the retinal motions accounts for performance under both projection systems, suggesting that this early noise forms the primary limitation on 3D discrimination performance.

Does the visual system exploit projective geometry to help solve the motion correspondence problem?

Projective geometry determines how the retinal image of an object deforms as it moves through three-dimensional space. Does the visual system use constraints derived from this information, such as rigidity, to aid the tracking of moving objects? A novel psychophysical technique is introduced for assessing which of two competing motion transformations is 'preferred' by the visual system, in a two-frame sequence. In the first experiment, relative preference strengths for translations parallel and perpendicular to the major axis of a wire-frame object were measured by pitting the two against each other. It was found that parallel translations were preferred to perpendicular ones. On the basis of these data a proximity measure for normalising different transformations, independent of any effects of figural similarity, was developed. In the second experiment, two wire-frame planar structures were used to pit one of five transformations (rotation, expansion, vertical expansion, shear and random jitter) against a translation. Preference strength was measured as the translation distance at which the transformation and the translation were perceived with equal frequency. The PSEs were found to collapse on to a single line when plotted against the proximity magnitude, with the exception of a residual preference for pure translation over all other transformations. In general, these results suggest that preference strength for moving wire-frame figures is determined primarily by the proximity of local features on the displacing contour, with little regard for the projective shape transformation.

Upper displacement limits for spatially broadband patterns containing bandpass noise.

How is the spatial-frequency content of a moving broadband pattern analysed by the visual system? Observers were asked to discriminate the direction of motion in random-noise patterns containing equal energy in each two-dimensional octave band. Uncorrelated noise could be introduced into either low- or high-frequency bands in order to force the visual system to rely on the outputs of putative mechanisms tuned to a narrow frequency range of the stimulus. In two experiments the dependent measure was the magnitude of dmax, the largest discrete displacement whose direction could be discriminated reliably. It was found that dmax was unaffected by the presence of high-frequency noise reaching down to 0.67 c/deg, but that the task became impossible thereafter. In the case of low-frequency noise, dmax fell as the noise was moved up towards about 2 c/deg, at which point the task became impossible at any displacement. This pattern of results would be expected if the system were using information from the lowest signal frequencies in all conditions. In experiment 2, dmax was measured for stimuli in which the spectral position and quantity of high-frequency noise were manipulated. It was found that only noise spectrally-adjacent to the signal band has a detrimental effect on dmax. Three different single-filter models of motion detection each failed to provide a satisfactory account of the spatial-frequency range of good direction discrimination performance. Rather, the modelling shows that the visual system can access the outputs of a low-frequency channel when the noise is high and a high-frequency channel when the noise is low.

Biases in three-dimensional structure-from-motion arise from noise in the early visual system.

The projected pattern of retinal-image motion supplies the human visual system with valuable information about properties of the three-dimensional environment. How well three-dimensional properties can be recovered depends both on the accuracy with which the early motion system estimates retinal motion, and on the way later processes interpret this retinal motion. Here we combine both early and late stages of the computational process to account for the hitherto puzzling phenomenon of systematic biases in three-dimensional shape perception. We present data showing how the perceived depth of a hinged plane ('an open book') can be systematically biased by the extent over which it rotates. We then present a Bayesian model that combines early measurement noise with geometric reconstruction of the three-dimensional scene. Although this model has no in-built bias towards particular three-dimensional shapes, it accounts for the data well. Our analysis suggests that the biases stem largely from the geometric constraints imposed on what three-dimensional scenes are compatible with the (noisy) early motion measurements. Given these findings, we suggest that the visual system may act as an optimal estimator of three-dimensional structure-from-motion.

The interaction of binocular disparity and motion parallax in determining perceived depth and perceived size.

Although binocular disparity and motion parallax are powerful cues for depth, neither, in isolation, can specify information about both object size and depth. It has been shown that information from both cues can be combined to specify the size, depth, and distance of an object in a scene (Richards, 1985 Journal of the Optical Society of America A 2 343-349). Experiments are reported in which natural viewing and physical stimuli have been used to investigate the nature of size and depth perception on the basis of disparity and parallax presented separately and together at a range of viewing distances. Observers adjusted the relative position of three bright LEDs, which were constrained to form a triangle in plan view with the apex pointing toward the observer, so its dimensions matched that of a standard held by the subject. With static monocular viewing, depth settings were inaccurate and erratic. When both cues were present together accuracy increased and the perceptual outcome was consistent with an averaging of the information provided by both cues. When an apparent bias evident in the observers' responses (the tendency to under-estimate the size of the standard) was taken into account, accuracy was high and size and depth constancy were close to 100%. In addition, given this assumption, the same estimate of viewing distance was used to scale size and depth estimates.

Contrast masking reveals spatial-frequency channels in stereopsis.

Yang and Blake (1991 Vision Research 31 1177-1189) investigated depth detection in stereograms containing spatially narrow-band signal and noise energies. The resulting masking functions led them to conclude that stereo vision was subserved by only two channels peaking at 3 and 5 cycles deg-1. Glennerster and Parker (1997 Vision Research 37 2143-2152) re-analysed these data, taking into account the relative attenuation of low- and high-frequency noise masks as a consequence of the modulation transfer function (MTF) of the early visual system. They transformed the data using an estimated MTF and found that peak masking was always at the signal frequency across a 2.8 octave range. Here we determine the MTF of the early visual system for individual subjects by measuring contrast thresholds in a 2AFC orientation-discrimination task (horizontal vs vertical) using band-limited stimuli presented in a 7 deg x 7 deg window at 4 deg eccentricity. The filtered stimuli had a bandwidth of 1.5 octaves in frequency and 15 degrees in orientation at half-height. In the subsequent stereo experiment, the same (vertical) filters were used to generate both signal and noise bands. The noise was binocularly uncorrelated and scaled by each subject's MTF. Subjects performed a 2AFC depth-discrimination task (crossed vs uncrossed disparity) to determine threshold signal contrast as a function of signal and mask frequency. The resulting functions showed that peak masking was at the signal frequency over the three octave range tested (0.4-3.2 cycles deg-1). Comparison with simple luminance-masking data from experiments with similar stimuli shows that bandwidths for stereo masking are considerably larger. These data suggest that there are multiple bandpass channels feeding into stereopsis but that their characteristics differ from luminance channels in pattern vision.

Effects of dot density, patch size and contrast on the upper spatial limit for direction discrimination in random-dot kinematograms.

Two-frame random-dot kinematograms (RDKs) of different dot density, area and contrast were used to study the spatial properties of the human visual motion system. It was found that the maximum spatial displacement at which observers could reliably discriminate the direction of motion (dmax) increased gradually by a factor of up to 6.4 as dot density was decreased from 50 to 0.025% for high Michelson contrast (0.997) stimuli. As stimulus area was reduced from 645 deg2, this trend gradually disappeared so that by a stimulus area of 2.56 deg2, there was no effect of density upon dmax. A further experiment investigated the effects of reducing Michelson contrast from 0.77 to 0.2 on dmax over this same range of dot densities. It was found that at the highest densities, dmax declined as contrast was reduced. Furthermore, for contrasts at and below 0.4, dmax was invariant of density over the range 50-5%. These results can be accounted for by the fact that both reducing contrast, while keeping density fixed, and reducing density, while maintaining a fixed high contrast, reduce the stimulus mean luminance. For all contrasts, decreasing density below 5% led to an increase in dmax. However, the rate of this increase was slower for the lower contrast stimuli. A two-stage model based on bandpass filtering followed by an informationally limited motion detection stage is proposed and shown to provide a good account of these data.

Independent processing across spatial frequency in moving broadband patterns.

The aim of the experiments was to discover whether the visual system has independent access to motion information at different spatial scales when presented with a broadband stimulus. Subjects were required to discriminate between a pair of two-frame motion sequences, one containing a coherently displacing pattern and the other containing a pattern with high-frequency noise. The stimuli were either narrowband (1 octave) or broadband (6 octaves spanning 0.23-15.0 cycles deg-1) and their power spectra were either flat or followed a 1/f2 function. For the broadband stimuli, noise was introduced cumulatively into increasingly lower frequencies. For the narrowband stimuli, noise was introduced into the same frequency band as the signal. All stimuli could be defined by the lowest noise frequency (nl) they contained. For each stimulus, the largest spatial displacement across the two frames at which the task could be performed was measured (dmax). For the narrowband stimuli, dmax increased as nl was lowered. This was true over the entire frequency range for the 1/f2 stimuli, though the task became impossible for the flat-spectrum stimuli at the lowest frequencies. This is attributed to the very low contrast of these latter stimuli. The dmax values for the broadband stimuli tended to shadow those of the narrowband stimuli with the equivalent values of nl being around 25% lower. The data were modelled by spatiotemporally filtering the stimuli and considering the amount of directional power in the signal and noise sequences. The results suggest that there must be multiple spatial-frequency channels in operation, and that for broadband patterns the visual system has perceptual access to these individual channel outputs, utilising different filters depending on the task requirements.

Motion detection is limited by element density not spatial frequency.

Two-frame random-element kinematograms were used to study the matching algorithm employed by the visual system to keep track of moving elements. Previous data have shown that the maximum spatial displacement detectable (dmax) for random-dot kinematogram stimuli increases both with increasing dot size and with decreasing centre frequency for spatially band-pass kinematograms. Both of these findings could be explained by either (i) a matching algorithm sensitive to the number of false targets in the display (informational limit) or (ii) spatial-frequency tuned sensors hardwired for detecting displacements of a constant proportion of their preferred frequency (phase-based limit). The present experiment was designed to differentiate between these alternative explanations. The stimuli were band-pass filtered (difference-of-Gaussian) random-dot patterns. The combination of six dot densities and three filter sizes produced 18 experimental conditions and allowed independent control of the spectral content and filtered-element density of the stimuli. When the dot density was high, dmax was larger for the coarse-filtered stimuli, as predicted by both theories. There was also a critical dot density for each filter size, above which dmax was constant but below which dmax rose sharply. This critical density was higher for fine-filtered stimuli such that at the lowest dot density of 0.025%, dmax was constant for all filter sizes. In support of the informational limit model, dmax was found to be directly proportional to the two-dimensional spacing of filtered elements. In contrast, dmax varied from 0.6 to 8.5 cycles of the stimulus peak frequency, suggesting that a phase-based model of motion detection cannot account for the results.

Two-dimensional constraints on three-dimensional structure from motion tasks.

Can humans recover metric structure from motion sequences or, as has been claimed by Todd and Bressan [(1990) Perception & Psychophysics, 48, 419-430], are they limited to recovering only relief structure? Two experiments were carried out to investigate this question. In a metric-structure task, the angular thresholds for discriminating two rotating bi-planar structures were approximately 91 deg. By contrast, in a relief-structure task, the angular thresholds for discriminating a planar from a non-planar structure, both undergoing simple rotational motion, were only approximately 11 deg. A computational model is proposed to examine the image motion sensitivity required to perform discriminations of both three-dimensional metric and relief structure from motion. When the experimental data were re-plotted in terms of this two-dimensional sensitivity, the thresholds were found to be the same for both tasks. This finding is related to the model's revelation that recovering metric structure from motion is inherently more noise-sensitive than is recovering relief structure from motion. The conclusion is that the differences in angular thresholds reflect the differing nature of the two tasks. There is no evidence that the visual processes themselves are preferentially sensitive to non-metric over metric structure from motion.