Psychophysical evidence for two routes to suppression before binocular summation of signals in human vision.

Visual mechanisms in primary visual cortex are suppressed by the superposition of gratings perpendicular to their preferred orientations. A clear picture of this process is needed to (i) inform functional architecture of image-processing models, (ii) identify the pathways available to support binocular rivalry, and (iii) generally advance our understanding of early vision. Here we use monoptic sine-wave gratings and cross-orientation masking (XOM) to reveal two cross-oriented suppressive pathways in humans, both of which occur before full binocular summation of signals. One is a within-eye (ipsiocular) pathway that is spatially broadband, immune to contrast adaptation and has a suppressive weight that tends to decrease with stimulus duration. The other pathway operates between the eyes (interocular), is spatially tuned, desensitizes with contrast adaptation and has a suppressive weight that increases with stimulus duration. When cross-oriented masks are presented to both eyes, masking is enhanced or diminished for conditions in which either ipsiocular or interocular pathways dominate masking, respectively. We propose that ipsiocular suppression precedes the influence of interocular suppression and tentatively associate the two effects with the lateral geniculate nucleus (or retina) and the visual cortex respectively. The interocular route is a good candidate for the initial pathway involved in binocular rivalry and predicts that interocular cross-orientation suppression should be found in cortical cells with predominantly ipsiocular drive.

Performance data indicate summation for pictorial depth-cues in slanted surfaces.

Over recent years much has been learned about the way in which depth cues are combined (e.g. Landy et al.. 1995). The majority of this work has used subjective measures, a rating scale or a point of subjective equality, to deduce the relative contributions of different cues to perception. We have adopted a very different approach by using two interval forced-choice (21FC) performance measures and a signal processing framework. We performed summation experiments for depth cue increment thresholds between pairs of pictorial depth cues in displays depicting slanted planar surfaces made from arrays of circular 'contrast' elements. Summation was found to be ideal when size-gradient was paired with contrast-gradient for a wide range of depth-gradient magnitudes in the null stimulus. For a pairing of size-gradient and linear perspective, substantial summation (> 1.5 dB) was found only when the null stimulus had intermediate depth gradients; when flat or steeply inclined surfaces were depicted, summation was diminished or abolished. Summation was also abolished when one of the target cues was (i) not a depth cue, or (ii) added in conflict. We conclude that vision has a depth mechanism for the constructive combination of pictorial depth cues and suggest two generic models of summation to describe the results. Using similar psychophysical methods. Bradshaw and Rogers (1996) revealed a mechanism for the depth cues of motion parallax and binocular disparity. Whether this is the same or a different mechanism from the one reported here awaits elaboration.

Adaptation and gain pool summation: alternative models and masking data.

Foley [J. Opt. Soc. Am. A 11 (1994) 1710] has proposed an influential psychophysical model of masking in which mask components in a contrast gain pool are raised to an exponent before summation and divisive inhibition. We tested this summation rule in experiments in which contrast detection thresholds were measured for a vertical 1 c/deg (or 2 c/deg) sine-wave component in the presence of a 3 c/deg (or 6 c/deg) mask that had either a single component oriented at -45 degrees or a pair of components oriented at +/-45 degrees. Contrary to the predictions of Foley's model 3, we found that for masks of moderate contrast and above, threshold elevation was predicted by linear summation of the mask components in the inhibitory stage of the contrast gain pool. We built this feature into two new models, referred to as the early adaptation model and the hybrid model. In the early adaptation model, contrast adaptation controls a threshold-like nonlinearity on the output of otherwise linear pathways that provide the excitatory and inhibitory inputs to a gain control stage. The hybrid model involves nonlinear and nonadaptable routes to excitatory and inhibitory stages as well as an adaptable linear route. With only six free parameters, both models provide excellent fits to the masking and adaptation data of Foley and Chen [Vision Res. 37 (1997) 2779] but unlike Foley and Chen's model, are able to do so with only one adaptation parameter. However, only the hybrid model is able to capture the features of Foley's (1994) pedestal plus orthogonal fixed mask data. We conclude that (1) linear summation of inhibitory components is a feature of contrast masking, and (2) that the main aftereffect of spatial adaptation on contrast increment thresholds can be assigned to a single site.

Independent detectors for expansion and rotation, and for orthogonal components of deformation.

It is well known that optic flow--the smooth transformation of the retinal image experienced by a moving observer--contains valuable information about the three-dimensional layout of the environment. From psychophysical and neurophysiological experiments, specialised mechanisms responsive to components of optic flow (sometimes called complex motion) such as expansion and rotation have been inferred. However, it remains unclear (a) whether the visual system has mechanisms for processing the component of deformation and (b) whether there are multiple mechanisms that function independently from each other. Here, we investigate these issues using random-dot patterns and a forced-choice subthreshold summation technique. In experiment 1, we manipulated the size of a test region that was permitted to contain signal and found substantial spatial summation for signal components of translation, expansion, rotation, and deformation embedded in noise. In experiment 2, little or no summation was found for the superposition of orthogonal pairs of complex motion patterns (eg expansion and rotation), consistent with probability summation between pairs of independent detectors. Our results suggest that optic-flow components are detected by mechanisms that are specialised for particular patterns of complex motion.

Broad direction bandwidths for complex motion mechanisms.

Growing evidence from psychophysics and single-unit recordings suggests specialised mechanisms in the primate visual system for the detection of complex motion patterns such as expansion and rotation. Here we used a subthreshold summation technique to determine the direction tuning functions of the detecting mechanisms. We measured thresholds for discriminating noise and signal+noise for pairs of superimposed complex motion patterns (signal A and B) carried by random-dot stimuli in a circular 5 degrees field. For expansion, rotation, deformation and translation we found broad tuning functions approximated by cos(d), where d is the difference in dot directions for signal A and B. These data were well described by models in which either: (a) cardinal mechanisms had direction bandwidths (half-widths) of around 60 degrees; or (b) the number of mechanisms was increased and their half-width was reduced to about 40 degrees. When d=180 degrees we found summation to be greater than probability summation for expansion, rotation and translation, consistent with the idea that mechanisms for these stimuli are constructed from subunits responsive to relative motion. For deformation, however, we found sensitivity declined when d=180 degrees, suggesting antagonistic input from directional subunits in the deformation mechanism. This is a necessary property for a mechanism whose job is to extract the deformation component from the optic flow field.

Spatial coherence does not affect contrast discrimination for multiple Gabor stimuli.

Gestalt grouping rules imply a process or mechanism for grouping together local features of an object into a perceptual whole. Several psychophysical experiments have been interpreted as evidence for constrained interactions between nearby spatial filter elements and this has led to the hypothesis that element linking might be mediated by these interactions. A common tacit assumption is that these interactions result in response modulation which disturbs a local contrast code. We addressed this possibility by performing contrast discrimination experiments using two-dimensional arrays of multiple Gabor patches arranged either (i) vertically, (ii) in circles (coherent conditions), or (iii) randomly (incoherent condition), as well as for a single Gabor patch. In each condition, contrast increments were applied to either the entire test stimulus (experiment 1) or a single patch whose position was cued (experiment 2). In experiment 3, the texture stimuli were reduced to a single contour by displaying only the central vertical strip. Performance was better for the multiple-patch conditions than for the single-patch condition, but whether the multiple-patch stimulus was coherent or not had no systematic effect on the results in any of the experiments. We conclude that constrained local interactions do not interfere with a local contrast code for our suprathreshold stimuli, suggesting that, in general, this is not the way in which element linking is achieved. The possibility that interactions are involved in enhancing the detectability of contour elements at threshold remains unchallenged by our experiments.

Probability summation for multiple patches of luminance modulation.

When components of a compound pattern stimulate different visual mechanisms, psychophysical performance typically improves by a small amount consistent with probability summation amongst independent detectors. Here we extend previous summation experiments by (i) plotting full psychometric functions; and (ii) using compound stimuli with components that varied in up to three stimulus dimensions: spatial frequency (1, 4, 5 or 11 c/deg), orientation (0 degrees, +/-45 degrees ), and position. Stimulus components were isolated circular sine-phase patches of grating centred on up to four corners of an imaginary square surrounding a fixation-point. Combinations of component patches produced compound stimuli made from up to 16 components that differed in various combinations of the three stimulus dimensions. Other than when the spatial frequency was 11 c/deg, results were well described using: (i) probabilistic summation of individual psychometric functions; (ii) the Quick pooling formula; and (iii) the signal detection analysis for 2IFC developed by Tyler and Chen (2000) [Signal detection theory in the 2AFC paradigm: attention, channel uncertainty and probability summation (under review)]. We conclude that in general, nonlinear spatial summation is consistent with probabilistic summation across independent detecting mechanisms that vary in spatial frequency (a range of at least 1-5 c/deg), orientation (a range of 90 degrees ) and position (a range of at least 24 cycles at 4 c/deg). In further experiments, results were found to be consistent with probability summation for pairs of orthogonally oriented step-edge stimuli and a matrix of randomly oriented 11 c/deg sine-wave patches. This casts doubt on the generality of a recent suggestion that local interactions between colinearly oriented detectors within a spatial neighbourhood of around four cycles may contribute to nonlinear spatial summation [Bonneh & Sagi, 1998; Vision Research, 38, 3541-3553].

A model of human pattern perception: association fields for adaptive spatial filters.

Visual neurons in the primary visual cortex 'look' at the retinal image through a four-dimensional array of spatial receptive fields (filter-elements): two spatial dimensions and, at each spatial location, two Fourier dimensions of spatial frequency and orientation. In general, visual objects activate filter-elements along each of these dimensions, suggesting a need for some kind of linking mechanism that determines whether two or more filter-elements are responding to the same or different contours or objects. In the spatial domain, a (spatial) association field between filter-elements, arranged to form first-order curves, has been inferred as a flexible method by which different parts of extended (luminance) contours become associated (Field et al., 1993). Linking has also been explored between filters selective for different regions in Fourier space (e.g. Georgeson and Meese, 1997). Perceived structure of stationary plaids suggests that spatial filtering is adaptive: synthetic filters can be created by the linear summation of basis-filters across orientation or spatial frequency in a stimulus-dependent way. For example, a plaid with a pair of sine-wave components at +/-45 deg looks like a blurred checkerboard; a structure that can be understood if features are derived after linear summation of spatial filters at different orientations. However, the addition of an oblique third-harmonic component causes the plaid to perceptually segment into overlapping oblique contours. This result can be understood if filters are summed across spatial frequency, but, in this case, treated independently across orientation. In the present paper, the architecture of an association field is proposed to permit linking and segmentation of filter-elements across spatial frequency and orientation. Three types of link are proposed: (1) A chain of constructive links around sites of common spatial frequency but different orientation, to promote binding of filters across orientation; (2) Constructive links between sites with common orientation but different spatial frequency, to promote binding of filters across spatial frequency; (3) Long-range links between sites of common spatial frequency but different orientation, whose activation and role are determined by activity in a higher spatial frequency band. A model employing the proposed network of links is consistent with at least six previously reported effects on the perception of briefly presented stationary plaids.

Adaptive filtering in spatial vision: evidence from feature marking in plaids.

Much evidence shows that early vision employs an array of spatial filters tuned for different spatial frequencies and orientations. We suggest that for moderately low spatial frequencies these preliminary filters are not treated independently, but are used to perform grouping and segmentation in the patchwise Fourier domain. For example, consider a stationary plaid made from two superimposed sinusoidal gratings of the same contrast and spatial frequency oriented +/- 45 degrees from vertical. Most of the energy in a wavelet-like (e.g. simple-cell) transform of this stimulus is in the oblique orientations, but typically it looks like a compound structure containing blurred vertical and horizontal edges. This checkerboard structure corresponds with the locations of zero crossings in the output of an isotropic (circular) filter, synthesised from the linear sum of a set of oriented basis-filters (Georgeson, 1992 Proceedings of the Royal Society of London, Series B 249 235-245). However, the addition of a third harmonic in square-wave phase causes almost complete perceptual segmentation of the plaid into two overlapping oblique gratings. Here we confirm this result psychophysically using a feature-marking technique, and argue that this perceptual segmentation cannot be understood in terms of the zero crossings marked in the output of any static linear filter that is sensitive to all of the plaid's components. If it is assumed that zero crossings or similar are an appropriate feature-primitive in human vision, our results require a flexible process that combines and segments early basis-filters according to prevailing image conditions. Thus, we suggest that combination and segmentation of spatial filters in the patchwise Fourier domain underpins the perceptual segmentation observed in our experiments. Under this kind of image-processing scheme, registration across spatial scales occurs at the level of spatial filters, before features are extracted. This contrasts with many previous schemes where feature correspondence is required between spatial edge-maps at different spatial scales.

Computation of surface slant from optic flow: orthogonal components of speed gradient can be combined.

In previous work [Meese et al. (1995). Vision Research, 35, 2879-2888)] we showed that one-dimensional (1D) speed gradients are sufficient to produce a compelling impression of surface slant. Summing a 1D vertical shearing gradient or, less intuitively, a 1D horizontal shearing gradient with a random field of horizontally translating dots produces perceived slant about a horizontal axis. Similarly, a 1D vertical or horizontal compression gradient produces perceived slant about a vertical axis. Appropriately combining orthogonal 1D shears or compressions produces a purely deforming flow pattern. Here we asked whether both the vertical and horizontal components in such a stimulus contribute to perceived slant. Using a matching technique we found that for surfaces inclined about a vertical axis, this was indeed the case and that horizontal and vertical compression gradients contributed roughly equally to perceived slant. Similarly, for surfaces inclined about a horizontal axis, both vertical and horizontal shearing gradients contributed to the perceived slant, though here the horizontal gradient was given less weight than the vertical. We conclude that under appropriate conditions, the human visual system combines orthogonal speed gradients prior to the computation of slant from retinal flow.

Perception of stationary plaids: the role of spatial filters in edge analysis.

Orientation-tuned spatial filters in visual cortex are widely held to act as "orientation detectors", but our experiments on the perception of stationary two-dimensional (2-D) plaids require a new view. When two sinusoidal gratings at different orientations (say 1 c/deg, +/- 45 deg from vertical) are superimposed to form a standard plaid they do not, in general, look like two sets of oblique contours (diamonds) but more like a blurred checkerboard (squares) with vertical and horizontal edges, although the Fourier components are oblique. The pattern of edges seen in this plaid and others corresponds to the zero-crossings (ZCs) in the output of a circular filter, but adaptation and masking experiments suggest that oriented filters are being summed to emulate circular filtering, before ZC analysis. At low contrasts or after adaptation to an intermediate orientation, the combining of filters can fail or be "broken", and the diamond structure of the components is seen instead. Adding a low contrast third harmonic to one component in square-wave phase also changed the plaid's appearance from squares to diamonds, but adapting to the third harmonic enhanced the square appearance. Filters can evidently switch from combining across orientation to combining across spatial frequency. The combination stage of edge detection may involve variably weighted summing of oriented filters in monocular pathways, followed by a process that makes explicit the locations and orientations of features.

The tilt aftereffect in plaids and gratings: channel codes, local signs and "patchwise" transforms.

The perceived structure of a suprathreshold plaid made from two sinusoidal gratings tilted +/- 45 deg from vertical usually resembles a blurred checkerboard. Physically increasing the tilt of the components away from vertical elongates the pattern horizontally, yielding rectangular checks. We asked whether illusory tilt of the components induced by adaptation to a vertical grating would also give apparent elongation of the checks. Using a staircase method, we found that after adaptation the component orientations had to be set 3-4 deg closer to vertical to maintain a square, checkerboard-like appearance, implying that the +/- 45 deg plaid did appear elongated. This result suggests that the tilt aftereffect adaptation not only distorts the orientation of 1-D patterns but can also change the relative location of features, even when their orientation is not altered. This is not easy to explain within filter models that (often implicitly) assume each receptive field carries a fixed "local sign" indicating its position within the retinal array. We consider a representation in which localized patches of the image are encoded by the coefficients of a 2-D Fourier-like transform. With a simple subtractive model of adaptation, we show that the spatial information carried by each patch would be distorted in just the way observed for plaids and gratings. The fact that perceived structure is both distorted and coherent, not fragmented, after adaptation suggests an additional, more global process whereby local patches are combined to form a coherent, composite "neural image". We offer a simple principle that could establish this coherence. If the positions of local patches are adjusted so as to maximize the contrast energy of the composite image, then the spatial distortions of the patches (induced by selective adaptation) are carried through to the global structure of the composite. Thus, the conflict between channel codes and local signs is resolved, but perceptual distortion is the result.

Spatial filter combination in human pattern vision: channel interactions revealed by adaptation.

Above threshold, two superimposed sinusoidal gratings of the same spatial frequency (eg 1 cycle deg-1), of equal moderate contrast (eg C1 = C2 = 6%), and with orientations of +/- 45 degrees, usually look like a compound structure containing vertical and horizontal edges (ie a blurred checkerboard). These feature orientations are very different from the dominant filter orientations in a wavelet-type (eg simple-cell) transform of the stimulus, and so present a serious challenge to conventional models of orientation coding based on labelled linear filters. Previous experiments on perceived structure in static plaids have led to the view that the outputs of tuned spatial filters are combined in a stimulus-dependent way, before features such as edges are extracted. Here an adaptation paradigm was used to investigate the cross-channel interactions that appear to underlie the spatial-filter-combination process. Reported are two aftereffects of selective adaptation: (i) adaptation to a 1 cycle deg-1 plaid whose component orientations are intermediate to those in a 1 cycle deg-1 test plaid 'breaks' perceptual combination of the components in the test plaid; (ii) adapting to a 3 cycles deg-1 plaid whose component orientations match those in a 1 cycle deg-1 test plaid facilitates perceptual combination of the components in the test plaid. The results are taken as evidence that spatial channels remote from those most responsive to a test plaid play a crucial role in determining whether the test plaid segments or coheres perceptually.

On the relationship between deformation and perceived surface slant.

A compelling impression of surface slant is produced by random dot displays depicting deformation and translation alone. A simple model of slant estimation based upon deformation is shown to capture quantitatively both the perceived slant in this situation and the distortion in perceived slant produced when constant deformation is added to random dot displays depicting moving slanted surfaces. The results confirm that deformation provides a simple account of perceived slant.

Speed gradients and the perception of surface slant: analysis is two-dimensional not one-dimensional.

Motion parallax provides cues to the three-dimensional layout of a viewed scene and, in particular, to surface tilt and slant. For example, as a textured surface, inclined around a horizontal axis, translates horizontally relative to an observer's view point, then, in the absence of head and eye movements, the observer's retinal flow will contain a one-dimensional (1D) vertical speed gradient. The direction of this gradient indicates the direction of surface tilt, and its magnitude and sign can be used in calculating the magnitude and sign of the surface slant. Alternatively, the same retinal flow contains a 1D translating component, plus a two-dimensional (2D) component of rotation (curl), and a 2D component of deformation (def). On this view, the direction of surface tilt is related to the orientation of def and the magnitude and sign of the surface slant is related to the magnitude and sign of def. We used computer generated random dot patterns as stimuli to determine whether the human visual system employs a 1D analysis (i.e. 1D speed gradients) or a 2D analysis (i.e. deformation) of surface slant from motion parallax. Using a matching technique we found compelling impressions of slant when we vector summed a translation field with (i) vertical shear, horizontal shear or deformation (made from vertical and horizontal shear), but not rotation; and (ii) vertical compression, horizontal compression or deformation (made from vertical and horizontal compression), but much less so for expansion. In both cases, the first three conditions contain def, but the fourth does not, and the last three conditions contain 1D speed gradients orthogonal to the perceived axis of inclination, but the first one does not. Therefore, the results from the first and fourth conditions distinguish between the two processing strategies. They support the idea that surface slant is coded by combining both horizontal and vertical speed gradients in a way similar to the 2D differential invariant def and oppose the view that surface slant is encoded by a 1D analysis of motion in a direction orthogonal to the perceived axis of inclination. In a further experiment, we found essentially no effect of reducing the field size from 18 to 9 deg.

Phase-reversal discrimination in one and two dimensions: performance is limited by spatial repetition, not spatial frequency content.

Lawden [(1983) Vision Research, 23, 1451-1463] used vertical gratings containing two frequencies (F, nF) in phase discrimination (F + nF against F - nF) and compound detection (F + nF against F) experiments, where thresholds were measured by manipulating the contrast of the nF component. When n was varied, Lawden found a phase-plateau of moderate breadth where phase discrimination thresholds were about half of those measured in compound detection. I present the results of similar experiments, using one-dimensional (gratings) and two-dimensional (plaids). In a sine-plaid condition, the 1F grating was split into two 1F plaid components at +/- 45 deg from vertical while the nF component remained a vertical grating. In a square-wave plaid (SqW-plaid) condition the plaid components were square waves. For each of these conditions, the horizontal spatial repetition (SR) of the plaid is given by (F/square root of 2); it is half an octave lower than the spatial frequency (SF) of the oblique components but it is not represented in the stimulus spectrum. By plotting phase discrimination relative to compound detection a phase-plateau was found for all three conditions. When these data were plotted as a function of SF ratio (nF/F) the curves describing the two plaid conditions were found to be leftward translations of that describing the grating condition. However, when the results were plotted as a function of SR ratio (nF/SR), the three functions lay on top of each other. The finding that phase-reversal discrimination is not governed by the Fourier attributes of the stimulus per se, rules out an explanation in terms of a linear, broad-band, phase-sensitive mechanism. Rather, the results imply that information is combined across the set of SF- and orientation-tuned mechanisms before the decision variable. These interactions appear to be governed by the spatial (not Fourier) attributes of the luminance profile of the stimulus. A modified version of Bennett's [(1993) Perception & Psychophysics, 53, 292-304] phase discrimination model is presented as a post-hoc account of the data.

Induced motion may account for the illusory transformation of optic flow fields found by Duffy and Wurtz.

Duffy and Wurtz [(1993) Vision Research, 33, 1481-1490] found an illusory shift in the position of the focus of expansion (FOE) of random dot patterns when planar motion was superimposed on expanding radial motion. Subjects indicated that this illusory shift was in the direction of the planar motion. This is the opposite direction to a true shift in the FOE that is perceived when the planar motion is vector summed with the expanding motion. We account for this illusion by suggesting that planar motion induces opposite motion on the expanding dots which after vector summation produces the illusory shift in the FOE. We use a matching technique with a method of adjustment to measure induced motion and perceived FOE in moving random dot patterns and present the results in support of our assertion.

Using the standard staircase to measure the point of subjective equality: a guide based on computer simulations.

In experiments that measure a point of subjective equality, it is necessary to employ a psychophysical technique that measures the 50% point (P50) on the psychometric function. The standard staircase is an attractive candidate, because its simplicity makes it easy to understand and implement. However, Pentland (1980, Perception & Psychophysics, 28, 377-379) found his own maximum likelihood method (Best PEST) to be considerably less variable than the standard staircase. Here, computer simulations were used to study the effects of manipulating several parameters of the standard staircase in order to try to find a variant that would be competitive with the performance of Best PEST. A graphical summary of some of the standard staircase implementations that were found to be most stable is presented in order that investigators may choose easily the parameters best suited to their requirements. A comparison with an implementation of Pentland's Best PEST failed to replicate Pentland's finding, suggesting that investigators should have no misgivings about the standard staircase in experiments that measure a point of subjective equality.

Edge computation in human vision: anisotropy in the combining of oriented filters.

Above threshold, two superimposed sinusoidal gratings of the same spatial frequency (eg 1 cycle deg-1) and equal contrasts, and with orientations balanced around vertical, usually look like a compound structure containing vertical and horizontal edges. However, at large plaid angles (ie large differences between component orientations) and low plaid contrasts there is a tendency for the stimulus to appear as two overlapping gratings (component structure) with obliquely oriented edges. These dependencies of perceived spatial structure in plaids are incompatible with an edge-coding scheme that uses only circular filters to compute zero-crossings, but instead support the idea that different oriented filters can (compound percept) or cannot (component percept) be combined before edges are represented. Here, further evidence is presented in support of this hypothesis. Two-component plaid stimuli had plaid angles of 45 degrees or 90 degrees, and a range of plaid orientations (ie a range of orientations around which the plaid components were balanced). Observers indicated whether each stimulus was perceived as a compound or component structure for a range of plaid contrasts. In addition to angle and contrast effects, perceived spatial structure was also found to depend on plaid orientation: compound structures were perceived more often when the plaid components were balanced around the cardinal axes of the retina. It is suggested that the principles governing the combination of oriented-filter outputs might be learnt during the development of the visual system by using a Hebb-type rule: coactivated filters are more likely to combine their outputs when activated on future occasions. Given the prominence of vertical and horizontal orientations in a carpentered environment, this simple rule promotes a network that combines filters balanced around cardinal axes more readily than oblique axes, in agreement with the results.