United we sense, divided we fail: context-driven perception of ambiguous visual stimuli.

Ambiguous visual stimuli provide the brain with sensory information that contains conflicting evidence for multiple mutually exclusive interpretations. Two distinct aspects of the phenomenological experience associated with viewing ambiguous visual stimuli are the apparent stability of perception whenever one perceptual interpretation is dominant, and the instability of perception that causes perceptual dominance to alternate between perceptual interpretations upon extended viewing. This review summarizes several ways in which contextual information can help the brain resolve visual ambiguities and construct temporarily stable perceptual experiences. Temporal context through prior stimulation or internal brain states brought about by feedback from higher cortical processing levels may alter the response characteristics of specific neurons involved in rivalry resolution. Furthermore, spatial or crossmodal context may strengthen the neuronal representation of one of the possible perceptual interpretations and consequently bias the rivalry process towards it. We suggest that contextual influences on perceptual choices with ambiguous visual stimuli can be highly informative about the neuronal mechanisms of context-driven inference in the general processes of perceptual decision-making.

Depth cue combination in spontaneous eye movements.

Where we look when we scan visual scenes is an old question that continues to inspire both fundamental and applied research. Recently, it has been reported that depth is an important variable in driving eye movements: the directions of spontaneous saccades tend to follow depth gradients, or, equivalently, surface tilts (L. Jansen, S. Onat, & P. König, 2009; M. Wexler & N. Ouarti, 2008). This has been found to hold for both simple and complex scenes and for a variety of depth cues. However, it is not known whether saccades are aligned with individual depth cues, or with a combination of depth cues. If saccades do follow a combination of depth cues, then it is interesting to ask whether this combination follows the same rules as the well-studied case of depth cue combination in conscious perception. We showed subjects surfaces inclined in depth, in which perspective and binocular disparity cues specified different plane orientations, with different degrees of both small and large conflict between the two sets of cues. We recorded subjects' spontaneous saccades while they scanned the scene, as well as their reports of perceived plane orientation. We found that distributions of spontaneous saccade directions followed the same pattern of depth cue combination as perceived surface orientation: a weighted linear combination of cues for small conflicts, and cue dominance for large conflicts. The weights assigned to the cues varied considerably from one subject to the next but were strongly correlated for saccades and perception; moreover, for both perception and saccades, cue weights could be modified by manipulating cue reliability in a way compatible with Bayesian theories of optimal cue combination. We also measured vergence, which allowed us to calculate the orientation of the plane fitted to points scanned in depth. Contrary to perception and saccades, vergence was dominated by a single cue, binocular disparity.

Widespread fMRI activity differences between perceptual states in visual rivalry are correlated with differences in observer biases.

When observing bistable stimuli, the percept can change in the absence of changes in the stimulus itself. When intermittently presenting a bistable stimulus, the number of perceptual alternations can increase or decrease, depending on the duration of the period that the stimulus is removed from screen between stimulus presentations (off-period). Longer off-periods lead to stabilization of the percept, while short off-periods produce perceptual alternations. Here we compare fMRI brain activation across percept repetitions and alternations when observing an intermittently presented ambiguously rotating structure from motion sphere. In the first experimental session, subjects were requested to voluntarily control the percept into either a repeating or an alternating perceptual regime at a single off-period. In a consecutive session, subjects observed the sphere uninstructed, and reported alternations and repetitions. The behavioral data showed that there were marked individual biases for observing the sphere as either repeating or alternating. The fMRI data showed activation differences between alternating and repeating perceptual regimes in an extensive network that included parietal cortex, dorsal premotor area, dorsolateral prefrontal cortex, supplementary motor area, insula, and cerebellum. However, these activation differences could all be explained by intersubject differences in the bias for one of the two perceptual regimes. The stronger the bias was for a particular perceptual regime, the less activation and vice versa. We conclude that widespread activation differences between perceptual regimes can be accounted for by differences in the perceptual bias for one of the two regimes.

Early interactions between neuronal adaptation and voluntary control determine perceptual choices in bistable vision.

At the onset of bistable stimuli, the brain needs to choose which of the competing perceptual interpretations will first reach awareness. Stimulus manipulations and cognitive control both influence this choice process, but the underlying mechanisms and interactions remain poorly understood. Using intermittent presentation of bistable visual stimuli, we demonstrate that short interruptions cause perceptual reversals upon the next presentation, whereas longer interstimulus intervals stabilize the percept. Top-down voluntary control biases this process but does not override the timing dependencies. Extending a recently introduced low-level neural model, we demonstrate that percept-choice dynamics in bistable vision can be fully understood with interactions in early neural processing stages. Our model includes adaptive neural processing preceding a rivalry resolution stage with cross-inhibition, adaptation, and an interaction of the adaptation levels with a neural baseline. Most importantly, our findings suggest that top-down attentional control over bistable stimuli interacts with low-level mechanisms at early levels of sensory processing before perceptual conflicts are resolved and perceptual choices about bistable stimuli are made.

Depth cues, rather than perceived depth, govern vergence.

We studied the influence of perceived surface orientation on vergence accompanying a saccade while viewing an ambiguous stimulus. We used the slant rivalry stimulus, in which perspective foreshortening and disparity specified opposite surface orientations. This rivalrous configuration induces alternations of perceived surface orientation, while the slant cues remain constant. Subjects were able to voluntarily control their perceptual state while viewing the ambiguous stimulus. They were asked to make a saccade across the perceived slanted surface. Our data show that vergence responses closely approximated the vergence response predicted by the disparity cue, irrespective of voluntarily controlled perceived orientation. However, comparing the data obtained while viewing the ambiguous stimulus with data from an unambiguous stimulus condition (when disparity and perspective specified similar surface orientations) revealed an effect of perspective cues on vergence. Collectively our results show that depth cues rather than perceived depth govern vergence.

Percept-choice sequences driven by interrupted ambiguous stimuli: a low-level neural model.

Existing neural explanations of spontaneous percept switching under steady viewing of an ambiguous stimulus do not fit the fact that stimulus interruptions cause the same percept to reappear across many ON/OFF cycles. We present a simple neural model that explains the observed behavior and predicts several more complicated percept sequences, without invoking any "high-level" decision making or memory. Percept choice at stimulus onset, which differs fundamentally from standard percept switching, depends crucially on a hitherto neglected interaction between local "shunting" adaptation and a near-threshold neural baseline. Stimulus ON/OFF timing then controls the generation of repeating, alternating, or more complex choice sequences. Our model also explains "priming" versus "habituation" effects on percept choice, reinterprets recent neurophysiological data, and predicts the emergence of hysteresis at the level of percept sequences, with occasional noise-induced sequence "hopping."

Direct extraction of curvature-based metric shape from stereo by view-modulated receptive fields.

Any computation of metric surface structure from horizontal disparities depends on the viewing geometry, and analysing this dependence allows us to narrow down the choice of viable schemes. For example, all depth-based or slant-based schemes (i.e. nearly all existing models) are found to be unrealistically sensitive to natural errors in vergence. Curvature-based schemes avoid these problems and require only moderate, more robust view-dependent corrections to yield local object shape, without any depth coding. This fits the fact that humans are strikingly insensitive to global depth but accurate in discriminating surface curvature. The latter also excludes coding only affine structure. In view of new adaptation results, our goal becomes to directly extract retinotopic fields of metric surface curvatures (i.e. avoiding intermediate disparity curvature). To find a robust neural realisation, we combine new exact analysis with basic neural and psychophysical constraints. Systematic, step-by-step 'design' leads to neural operators which employ a novel family of 'dynamic' receptive fields (RFs), tuned to specific (bi-)local disparity structure. The required RF family is dictated by the non-Euclidean geometry that we identify as inherent in cyclopean vision. The dynamic RF-subfield patterns are controlled via gain modulation by binocular vergence and version, and parameterised by a cell-specific tuning to slant. Our full characterisation of the neural operators invites a range of new neurophysiological tests. Regarding shape perception, the model inverts widely accepted interpretations: It predicts the various types of errors that have often been mistaken for evidence against metric shape extraction.

Attentional control over either of the two competing percepts of ambiguous stimuli revealed by a two-parameter analysis: means do not make the difference.

We studied distributions of perceptual rivalry reversals, as defined by the two fitted parameters of the Gamma distribution. We did so for a variety of bi-stable stimuli and voluntary control exertion tasks. Subjects' distributions differed from one another for a particular stimulus and control task in a systematic way that reflects a constraint on the describing parameters. We found a variety of two-parameter effects, the most important one being that distributions of subjects differ from one another in the same systematic way across different stimuli and control tasks (i.e., a fast switcher remains fast across all conditions in a parameter-specified way). The cardinal component of subject-dependent variation was not the conventionally used mean reversal rate, but a component that was oriented-for all stimuli and tasks-roughly perpendicular to the mean rate. For the Necker cube, we performed additional experiments employing specific variations in control exertion, suggesting that subjects have to a considerable extent independent control over the reversal rate of either of the two competing percepts.

Colour helps to solve the binocular matching problem.

The spatial differences between the two retinal images, called binocular disparities, can be used to recover the three-dimensional (3D) aspects of a scene. The computation of disparity depends upon the correct identification of corresponding features in the two images. Understanding what image features are used by the brain to solve this binocular matching problem is an important issue in research on stereoscopic vision. The role of colour in binocular vision is controversial and it has been argued that colour is ineffective in achieving binocular vision. In the current experiment subjects were required to indicate the amount of perceived depth. The stimulus consisted of an array of fronto-parallel bars uniformly distributed in a constant sized volume. We studied the perceived depth in those 3D stimuli by manipulating both colour (monochrome, trichrome) and luminance (congruent, incongruent). Our results demonstrate that the amount of perceived depth was influenced by colour, indicating that the visual system uses colour to achieve binocular matching. Physiological data have revealed cortical cells in macaque V2 that are tuned both to binocular disparity and to colour. We suggest that one of the functional roles of these cells may be to help solve the binocular matching problem.

Voluntary control and the dynamics of perceptual bi-stability.

Voluntary control and conscious perception seem to be related: when we are confronted with ambiguous images we are in some cases and to some extent able to voluntarily select a percept. However, to date voluntary control has not been used in neurophysiological studies on the correlates of conscious perception, presumably because the dynamic of perceptual reversals was not suitable. We exposed the visual system to four ambiguous stimuli that instigate bi-stable perception: slant rivalry, orthogonal grating rivalry, house-face rivalry, and Necker cube rivalry. In the preceding companion paper [van Ee, R. (2005). Dynamics of perceptual bi-stability for stereoscopic slant rivalry and a comparison with grating, house-face, and Necker cube rivalry. Vision Research] we focussed on the temporal dynamics of the perceptual reversals. Here we examined the role of voluntary control in the dynamics of perceptual reversals. We asked subjects to attempt to hold percepts and to speed-up the perceptual reversals. The investigations across the four stimuli revealed qualitative similarities concerning the influence of voluntary control on the temporal dynamics of perceptual reversals. We also found differences. In comparison to the other rivalry stimuli, slant rivalry exhibits: (1) relatively long percept durations; (2) a relatively clear role of voluntary control in modifying the percept durations. We advocate that these aspects, alongside with its metrical (quantitative) aspects, potentially make slant rivalry an interesting tool in studying the neural underpinnings of visual awareness.

Motion direction, speed and orientation in binocular matching.

The spatial differences between the images seen by the two eyes, called binocular disparities, can be used to recover the volumetric (three-dimensional) aspects of a scene. The computation of disparity depends upon the correct identification of corresponding features in the two images. Understanding what image features are used by the brain to solve this matching problem is one of the main issues in stereoscopic vision. Many cortical neurons in visual areas V1 (ref. 2), MT (refs 3, 4) and MST (refs 5, 6) that are tuned to binocular disparity are also tuned to orientation, motion direction and speed. Although psychophysical work has shown that motion direction can facilitate binocular matching, the psychophysical literature on the role of orientation is mixed, and it has been argued that speed differences are ineffective in aiding correspondence. Here we use a different psychophysical paradigm to show that the visual system uses similarities in orientation, motion direction and speed to achieve binocular correspondence. These results indicate that cells that multiplex orientation, motion direction, speed and binocular disparity may help to solve the binocular matching problem.

Perceptual learning without feedback and the stability of stereoscopic slant estimation.

Subjects were examined for practice effects in a stereoscopic slant-estimation task involving surfaces that comprised a large portion of the visual field. In most subjects slant estimation was significantly affected by practice, but only when an isolated surface (an absolute disparity gradient) was present in the visual field. When a second, unslanted, surface was visible (providing a second disparity gradient and thereby also a relative disparity gradient) none of the subjects exhibited practice effects. Apparently, stereoscopic slant estimation is more robust or stable over time in the presence of a second surface than in its absence. In order to relate the practice effects, which occurred without feedback, to perceptual learning, results are interpreted within a cue-interaction framework. In this paradigm the contribution of a cue depends on its reliability. It is suggested that normally absolute disparity gradients contribute relatively little to perceived slant and that subjects learn to increase this contribution by utilizing proprioceptive information. It is argued that--given the limited computational power of the brain--a relatively small contribution of absolute disparity gradients in perceived slant enhances the stability of stereoscopic slant perception.

Occlusion junctions do not improve stereoacuity.

Occlusion geometry gives rise to interocular shifts in the positions of binocularly viewed contour junctions. Since these shifts do not give rise to normal binocular disparities, they have been called 'pseudodisparities'. Previous work has shown that the unmatched contour segments of a partially occluded contour at occlusion junctions can be used to recover the geometry of the occluding surface through the construction of 'illusory' contours. Here, experiments were performed to determine whether such junction shifts could enhance stereoscopic depth detection when the relative disparity between the contours was below threshold. Our results showed that stereoscopic depth detection does not improve when pseudodisparity is present. We conclude that the visual system is less sensitive to pseudodisparity than to conventional disparity information. We suggest that the primary role of pseudodisparity is to overcome conditions of camouflage.

Is there an interaction between perceived direction and perceived aspect ratio in stereoscopic vision?

In monocular vision, the horizontal/vertical aspect ratio (shape) of a frontoparallel rectangle can be based on the comparison of the perceived directions of the rectangle's edges. In binocular vision of a typical three-dimensional scene (when occlusions are present), this is not the case: Frontoparallel rectangles would be perceived in a distorted fashion if an observer were to base perceived aspect ratio on the perceived directions of the rectangle's edges. We psychophysically investigated stereoscopically perceived aspect ratios of frontoparallel occluding and occluded rectangles for various distances and fixation depths. We found that observers did not perceive the distortions that would be predicted on the basis of the above-mentioned comparison of the perceived visual directions of the edges of the rectangle. Our results strongly suggest that the mechanism that determines perceived aspect ratio is dissociated from the mechanism that determines perceived direction. The consequences of the findings for the Kanizsa, Poggendorff, and horizontal/vertical illusions are discussed.

Unconstrained stereoscopic matching of lines.

The computation of horizontal binocular disparities used in stereoscopic depth perception depends upon the identification of corresponding features in the two retinal images. In principle, binocular matching is a two-dimensional problem that considers matches in all possible meridians. Normally, constraints such as end points or crossing points limit the direction and magnitude of matches. If matching is unconstrained, such as is the case with long lines, it is completely ambiguous. Under these conditions the default match will be determined by the operating range, or upper disparity limit, of matchable vertical and horizontal disparities. We computed the operating range of vertical matches for stereoscopic depth as a function of line orientation. Our results suggest that the two-dimensional operating range is anisotropic for vertical and horizontal disparity and that unconstrained matches are not based upon either epipolar geometry or nearest neighbor constraints, but rather the mean of disparity estimates within the operating range for binocular matches. This operating range can be extended vertically when matches are constrained by image primitives.

The influence of large scanning eye movements on stereoscopic slant estimation of large surfaces.

The results of several experiments demonstrate that the estimated magnitude of perceived slant of large stereoscopic surfaces increases with the duration of the presentation. In these experiments, subjects were free to make eye movements. A possible explanation for the increase is that the visual system needs to scan the stimulus with eye movements (which take time) before it can make a reliable estimate of slant. We investigated the influence of large scanning eye movements on stereoscopic slant estimation of large surfaces. Six subjects estimated the magnitude of slant about the vertical or horizontal axis induced by large-field stereograms of which one half-image was transformed by horizontal scale, horizontal shear, vertical scale, vertical shear, divergence or rotation relative to the other half-image. The experiment was blocked in three sessions. Each session was devoted to one of the following fixation strategies: central fixation, peripheral (20 deg) fixation and active scanning of the stimulus. The presentation duration in each of the sessions was 0.5, 2 or 8 s. Estimations were done with and without a visual reference. The magnitudes of estimated slant and the perceptual biases were not significantly influenced by the three fixation strategies. Thus, our results provide no support for the hypothesis that the time used for the execution of large scanning eye movements explains the build-up of estimated slant with the duration of the stimulus presentation.

Horizontal and vertical disparity, eye position, and stereoscopic slant perception.

The slant of a stereoscopically defined surface cannot be determined solely from horizontal disparities or from derived quantities such as horizontal size ratio (HSR). There are four other signals that, in combination with horizontal disparity, could in principle allow an unambiguous estimate of slant: the vergence and version of the eyes, the vertical size ratio (VSR), and the horizontal gradient of VSR. Another useful signal is provided by perspective slant cues. The determination of perceived slant can be modeled as a weighted combination of three estimates based on those signals: a perspective estimate, a stereoscopic estimate based on HSR and VSR, and a stereoscopic estimate based on HSR and sensed eye position. In a series of experiments, we examined human observers' use of the two stereoscopic means of estimation. Perspective cues were rendered uninformative. We found that VSR and sensed eye position are both used to interpret the measured horizontal disparities. When the two are placed in conflict, the visual system usually gives more weight to VSR. However, when VSR is made difficult to measure by using short stimuli or stimuli composed of vertical lines, the visual system relies on sensed eye position. A model in which the observer's slant estimate is a weighted average of the slant estimate based on HSR and VSR and the one based on HSR and eye position accounted well for the data. The weights varied across viewing conditions because the informativeness of the signals they employ vary from one situation to another.

Perceived visual direction near an occluder.

When an opaque object occludes a more distant object, the two eyes often see different parts of the distant object. Hering's laws of visual direction make an interesting prediction for this situation: the part seen by both eyes should be seen in a different direction than the part seen by one eye. We examined whether this prediction holds by asking observers to align a vertical monocular line segment with a nearby vertical binocular segment. We found it necessary to correct the alignment data for vergence errors, which were measured in a control experiment, and for monocular spatial distortions, which were also measured in a control experiment. Settings were reasonably consistent with Hering's laws when the monocular and binocular targets were separated by 30 arcmin or more. Observers aligned the targets as if they were viewing them from one eye only when they were separated by 2 arcmin; this behavior is consistent with an observation reported by Erkelens and colleagues. The same behavior was observed when the segments were horizontal and when no visible occluder was present. Perceived visual direction when the two eyes see different parts of a distant target is assigned in a fashion that minimizes, but does not eliminate, distortions of the shape of the occluded object.