Binocular receptive-field construction in the primary visual cortex.

ON and OFF thalamic afferents from the two eyes converge in the primary visual cortex to form binocular receptive fields. The receptive fields need to be diverse to sample our visual world but also similar across eyes to achieve binocular fusion. It is currently unknown how the cortex balances these competing needs between receptive-field diversity and similarity. Our results demonstrate that receptive fields in the cat visual cortex are binocularly matched with exquisite precision for retinotopy, orientation/direction preference, orientation/direction selectivity, response latency, and ON-OFF polarity/structure. Specifically, the average binocular mismatches in retinotopy and ON-OFF structure are tightly restricted to 1/20 and 1/5 of the average receptive-field size but are still large enough to generate all types of binocular disparity tuning. Based on these results, we conclude that cortical receptive fields are binocularly matched with the high precision needed to facilitate binocular fusion while allowing restricted mismatches to process visual depth.

Short communication: Binocular rivalry dynamics during locomotion.

Locomotion has been shown to impact aspects of visual processing in both humans and animal models. In the current study, we assess the impact of locomotion on the dynamics of binocular rivalry. We presented orthogonal gratings, one contrast-modulating at 0.8 Hz (matching average step frequency) and the other at 3.2 Hz, to participants using a virtual reality headset. We compared two conditions: stationary and walking. We continuously monitored participants' foot position using tracking devices to measure the step cycle. During the walking condition, participants viewed the rivaling gratings for 60-second trials while walking on a circular path in a virtual reality environment. During the stationary condition, observers viewed the same stimuli and environment while standing still. The task was to continuously indicate the dominant percept via button press using handheld controllers. We found no significant differences between walking and standing for normalized dominance duration distributions, mean normalized dominance distributions, mean alternation rates, or mean fitted frequencies. Although our findings do not align with prior research highlighting distinctions in normalized dominance distributions between walking and standing, our study contributes unique evidence indicating that alternation rates vary across the step cycle. Specifically, we observed that the number of alternations is at its lowest during toe-off phases and reaches its peak at heel strike. This novel insight enhances our understanding of the dynamic nature of alternation patterns throughout the step cycle.

The role of pattern coherence in interocular grouping during binocular rivalry: Insights from individual differences.

Interocular grouping during binocular rivalry occurs when two images presented to each eye combine into a coherent pattern. The experience of interocular grouping is thought to be influenced by both eye-of-origin, which involves excitatory lateral connections among monocular neurons, and pattern coherence, which results from top-down intervention from higher visual areas. However, it remains unclear which factor plays a more significant role in the interocularly-grouped percepts during binocular rivalry. The current study employed an individual difference approach to investigate whether grouping dynamics are mainly determined by eye-of-origin or pattern coherence. We found that participants who perceived interocularly-driven coherent percepts for a longer duration also tended to experience longer periods of monocularly-driven coherent percepts. In contrast, participants who experienced non-coherent piecemeal percepts for an extended duration in conventional rivalry also had longer duration of non-coherent percepts in the interocular coherence setting. This individual differences in experiencing interocular grouping suggest that pattern coherence exerts a stronger influence on grouping dynamics during binocular rivalry compared to eye-of-origin factors.

Classification images for aerial images capture visual expertise for binocular disparity and a prior for lighting from above.

Using a novel approach to classification images (CIs), we investigated the visual expertise of surveyors for luminance and binocular disparity cues simultaneously after screening for stereoacuity. Stereoscopic aerial images of hedges and ditches were classified in 10,000 trials by six trained remote sensing surveyors and six novices. Images were heavily masked with luminance and disparity noise simultaneously. Hedge and ditch images had reversed disparity on around half the trials meaning hedges became ditch-like and vice versa. The hedge and ditch images were also flipped vertically on around half the trials, changing the direction of the light source and completing a 2 × 2 × 2 stimulus design. CIs were generated by accumulating the noise textures associated with "hedge" and "ditch" classifications, respectively, and subtracting one from the other. Typical CIs had a central peak with one or two negative side-lobes. We found clear differences in the amplitudes and shapes of perceptual templates across groups and noise-type, with experts prioritizing binocular disparity and using this more effectively. Contrariwise, novices used luminance cues more than experts meaning that task motivation alone could not explain group differences. Asymmetries in the luminance CIs revealed individual differences for lighting interpretation, with experts less prone to assume lighting from above, consistent with their training on aerial images of UK scenes lit by a southerly sun. Our results show that (i) dual noise in images can be used to produce simultaneous CI pairs, (ii) expertise for disparity cues does not depend on stereoacuity, (iii) CIs reveal the visual strategies developed by experts, (iv) top-down perceptual biases can be overcome with long-term learning effects, and (v) CIs have practical potential for directing visual training.

Temporal mechanisms in frontoparallel stereomotion revealed by individual differences analysis.

Masking experiments, using vertical and horizontal sinusoidal depth corrugations, have suggested the existence of more than two spatial-frequency disparity mechanisms. This result was confirmed through an individual differences approach. Here, using factor analytic techniques, we want to investigate the existence of independent temporal mechanisms in frontoparallel stereoscopic (cyclopean) motion. To construct stereomotion, we used sinusoidal depth corrugations obtained with dynamic random-dot stereograms. Thus, no luminance motion was present monocularly. We measured disparity thresholds for drifting vertical (up-down) and horizontal (left-right) sinusoidal corrugations of 0.4 cyc/deg at 0.25, 0.5, 1, 2, 4, 6, and 8 Hz. In total, we tested 34 participants. Results showed a small orientation anisotropy with lower thresholds for horizontal corrugations. Disparity thresholds as a function of temporal frequency were almost constant from 0.25 up to 1 Hz, and then they increased monotonically. Principal component analysis uncovered two significant factors for vertical and two for horizontal corrugations. Varimax rotation showed that one factor loaded from 0.25 to 1-2 Hz and a second factor from 2 to 4 to 8 Hz. Direct Oblimin rotation indicated a moderate intercorrelation of both factors. Our results suggest the possible existence of two somewhat interdependent temporal mechanisms involved in frontoparallel stereomotion.

Low-Latency Ocular Parallax Rendering and Investigation of Its Effect on Depth Perception in Virtual Reality.

With a demand for an immersive experience in virtual/augmented reality (VR/AR) displays, recent efforts have incorporated eye states, such as focus and fixation, into display graphics. Among these, ocular parallax, a small parallax generated by eye rotation, has received considerable attention for its impact on depth perception. However, the substantial latency of head-mounted displays (HMDs) has made it challenging to accurately assess its true effect during free eye movements. To address this issue, we propose a high-speed (360 Hz) and low-latency (4.8 ms) ocular parallax rendering system with a custom-built eye tracker. Using this proposed system, we conducted an investigation to determine the latency requirements necessary for achieving perceptually stable ocular parallax rendering. Our findings indicate that, in binocular viewing, ocular parallax rendering is perceived as significantly less stable than conventional rendering when the latency exceeds 43.72 ms at 1.3 D and 21.50 ms at 2.0 D. We also evaluated the effects of ocular parallax rendering on binocular fusion and monocular depth perception under free viewing conditions. The results demonstrated that ocular parallax rendering can enhance binocular fusion but has a limited impact on depth perception under monocular viewing conditions when latency is minimized.

Mean orientation discrimination based on proximal stimuli.

Ensemble perception refers to the ability to accurately and rapidly perceive summary statistical representations of specific features from a group of similar objects. However, the specific type of representation involved in this perception within a three-dimensional (3-D) environment remains unclear. In the context of perspective viewing with stereopsis, distal stimuli can be projected onto the retina as different forms of proximal stimuli based on their distances, despite sharing similar properties, such as object size and spatial frequency. This study aimed to investigate the effects of distal and proximal stimuli on the perception of summary statistical information related to orientation. In our experiment, we presented multiple Gabor patches in a stereoscopic environment, allowing us to measure the discrimination threshold of the mean orientation. The object size and spatial frequency were fixed for all patches regardless of depth. However, the physical angular size and absolute spatial frequency covaried with the depth. The results revealed the threshold elevation with depth expansion, especially when the patches formed two clusters at near and far distances, leading to large variations in their retinotopic representations. This finding indicates a minor contribution of similarity of the distal stimuli. Subsequent experiments demonstrated that the variability in physical angular size of the patches significantly influenced the threshold elevation in contrast to that of binocular disparity and absolute spatial frequency. These findings highlight the critical role of physical angular size variability in perceiving mean orientations within the 3-D space.

Absolute and relative disparity mechanisms revealed by an equivalent noise analysis.

The precision of stereopsis and vergence are ultimately limited by internal binocular disparity noise. Here we propose an equivalent noise model with both global and local internal disparity noises to provide a unified explanation of both absolute and relative disparity thresholds. To test this model, we developed a psychophysical procedure to measure the equivalent internal disparity noise by adding external disparity noise to random-Gabor-patch stereograms. We used the method of constant stimuli to measure the minimum and maximum disparity thresholds (Dmin and Dmax) for both absolute and relative disparity. Consistent with previous studies, we found that Dmin thresholds are substantially worse for absolute disparity than for relative disparity. We tested three relative disparity mechanisms: (1) the difference between the monocular separations of targets projecting to the two eyes; (2) the direct measurement of relative disparity; and (3) the difference of absolute disparities of targets. Computing the difference of absolute disparities when detecting relative disparity, Mechanism 3 cancels global noise, resulting in a much lower relative Dmin threshold, and provides a reasonable fit to the experimental data. We also found that the presence of as much as 2400 arcsec of external disparity noise does not appear to affect the Dmax threshold. This observation suggests that Dmax is implicated in a mechanism that disregards the disparity variance of individual items, relying instead on the average disparity across all items, supporting the depth model proposed in our previous study (Ding & Levi, 2021), which posits distinct mechanisms governing Dmin and Dmax thresholds.

Effects of binocular disparity on binocular luminance combination.

This study aimed to examine the effects of binocular disparity on binocular combination of brightness information coming from luminance increments and decrements. The point of subjective equality was determined by asking the observers to judge which stimulus appeared brighter-a bar stimulus with variable disparity or another stimulus with zero disparity. For the bar stimulus, the interocular luminance ratio was varied to trace an equal brightness curve. Binocular disparity had no effect on luminance increments presented on a gray or black background. In contrast, when luminance decrements were presented on a gray background, non-zero disparities elevated points of subjective equality for stimuli with interocular luminance differences. This means that the binocular brightness combination of the two monocular signals shifted from winner-take-all summation toward linear averaging. It has been argued that this effect may be caused by non-zero binocular disparities attenuating interocular suppression, which is deemed to operate normally with zero disparity.

Effect of 3-D depth structure, element size, and area containing elements on total-element overestimation phenomenon.

The number of elements distributed in a three-dimensional stimulus is overestimated compared to a two-dimensional stimulus when both stimuli have the same number of elements. We examined the effect of the properties of a three-dimensional stimulus (the number of overlapping stereo surfaces, size of the elements, and size of the area containing elements, on the overestimation phenomenon in four experiments. The two stimuli were presented side-by-side with the same diameters. Observers judged which of the three-dimensional standard and two-dimensional comparison had more elements. The results showed that (a) the overestimation phenomenon occurred for the three-dimensional standard stimuli, (b) the size of the areas affected the amount of overestimation, while the number of overlapping stereo surfaces and size of elements did not, and (c) the amount of overestimation increased when the stimuli included more than 100 elements. Implications of these findings were discussed in the framework of back-surface bias, occlusion, and disparity-processing interference models.

Transformations of sensory information in the brain suggest changing criteria for optimality.

Neurons throughout the brain modulate their firing rate lawfully in response to sensory input. Theories of neural computation posit that these modulations reflect the outcome of a constrained optimization in which neurons aim to robustly and efficiently represent sensory information. Our understanding of how this optimization varies across different areas in the brain, however, is still in its infancy. Here, we show that neural sensory responses transform along the dorsal stream of the visual system in a manner consistent with a transition from optimizing for information preservation towards optimizing for perceptual discrimination. Focusing on the representation of binocular disparities-the slight differences in the retinal images of the two eyes-we re-analyze measurements characterizing neuronal tuning curves in brain areas V1, V2, and MT (middle temporal) in the macaque monkey. We compare these to measurements of the statistics of binocular disparity typically encountered during natural behaviors using a Fisher Information framework. The differences in tuning curve characteristics across areas are consistent with a shift in optimization goals: V1 and V2 population-level responses are more consistent with maximizing the information encoded about naturally occurring binocular disparities, while MT responses shift towards maximizing the ability to support disparity discrimination. We find that a change towards tuning curves preferring larger disparities is a key driver of this shift. These results provide new insight into previously-identified differences between disparity-selective areas of cortex and suggest these differences play an important role in supporting visually-guided behavior. Our findings emphasize the need to consider not just information preservation and neural resources, but also relevance to behavior, when assessing the optimality of neural codes.

An Accumulating Neural Signal Underlying Binocular Rivalry Dynamics.

During binocular rivalry, conflicting images are presented one to each eye and perception alternates stochastically between them. Despite stable percepts between alternations, modeling suggests that neural signals representing the two images change gradually, and that the duration of stable percepts are determined by the time required for these signals to reach a threshold that triggers an alternation. However, direct physiological evidence for such signals has been lacking. Here, we identify a neural signal in the human visual cortex that shows these predicted properties. We measured steady-state visual evoked potentials (SSVEPs) in 84 human participants (62 females, 22 males) who were presented with orthogonal gratings, one to each eye, flickering at different frequencies. Participants indicated their percept while EEG data were collected. The time courses of the SSVEP amplitudes at the two frequencies were then compared across different percept durations, within participants. For all durations, the amplitude of signals corresponding to the suppressed stimulus increased and the amplitude corresponding to the dominant stimulus decreased throughout the percept. Critically, longer percepts were characterized by more gradual increases in the suppressed signal and more gradual decreases of the dominant signal. Changes in signals were similar and rapid at the end of all percepts, presumably reflecting perceptual transitions. These features of the SSVEP time courses are well predicted by a model in which perceptual transitions are produced by the accumulation of noisy signals. Identification of this signal underlying binocular rivalry should allow strong tests of neural models of rivalry, bistable perception, and neural suppression.SIGNIFICANCE STATEMENT During binocular rivalry, two conflicting images are presented to the two eyes and perception alternates between them, with switches occurring at seemingly random times. Rivalry is an important and longstanding model system in neuroscience, used for understanding neural suppression, intrinsic neural dynamics, and even the neural correlates of consciousness. All models of rivalry propose that it depends on gradually changing neural activity that on reaching some threshold triggers the perceptual switches. This manuscript reports the first physiological measurement of neural signals with that set of properties in human participants. The signals, measured with EEG in human observers, closely match the predictions of recent models of rivalry, and should pave the way for much future work.

The effect of training on sensitivity and stability of double fusion in Panum's limiting case.

Panum's limiting case is a phenomenon of monocular occlusion in binocular vision. This occurs when one object is occluded by the other object for one eye, but the two objects are both visible for the other eye. Although previous studies have found that vertical gradient of horizontal disparity and cue conflict are two important factors for double fusion, the effect of training on the sensitivity and stability of Panum's limiting case remains unknown. The current study trained 26 participants for 5 days with several of Panum's configurations (Gilliam, Frisby, and Wang series). The latency and duration of double fusion were recorded to examine the effects of training on sensitivity and stability of double fusion in Panum's limiting case. For each level of vertical gradient of horizontal disparity and cue conflict, the latency of double fusion decreased and the duration of double fusion increased with each additional training session. The results showed that vertical gradient of horizontal disparity and cue conflict interacted, and the duration of high cue conflict was significantly shorter than that of medium and low cue conflict for each level of vertical gradient of horizontal disparity. The findings suggest that there is an effect of training for vertical gradient of horizontal disparity and cue conflict in Panum's limiting case, and that the three factors jointly affect the sensitivity and stability of double fusion.

Coarse-to-fine interaction on perceived depth in compound grating.

To encode binocular disparity, the visual system uses a pair of left eye and right eye bandpass filters with either a position or a phase offset between them. Such pairs are considered to exit at multiple scales to encode a wide range of disparity. However, local disparity measurements by bandpass mechanisms can be ambiguous, particularly when the actual disparity is larger than a half-cycle of the preferred spatial frequency of the filter, which often occurs in fine scales. In this study, we investigated whether the visual system uses a coarse-to-fine interaction to resolve this ambiguity at finer scales for depth estimation from disparity. The stimuli were stereo grating patches composed of a target and comparison patterns. The target patterns contained spatial frequencies of 1 and 4 cycles per degree (cpd). The phase disparity of the low-frequency component was 0° (at the horopter), -90° (uncrossed), or 90° (crossed), and that of the high-frequency components was changed independent of the low-frequency disparity, in the range between -90° (uncrossed) and 90° (crossed). The observers' task was to indicate whether the target appeared closer to the comparison pattern, which always shared the disparity with the low-frequency component of the target. Regardless of whether the comparison pattern was a 1-cpd + 4-cpd compound or a 1-cpd simple grating, the perceived depth order of the target and the comparison varied in accordance with the phase disparity of the high-frequency component of the target. This effect occurred not only when the low-frequency component was at the horopter, but also when it contained a large disparity corresponding to one cycle of the high-frequency component (±90°). Our findings suggest a coarse-to-fine interaction in multiscale disparity processing, in which the depth interpretation of the high-frequency changes based on the disparity of the low-frequency component.

Integration and suppression interact in binocular vision.

Contingent on stereo compatibility, two images presented dichoptically can lead to either binocular integration, thus generating stable stereopsis, or interocular suppression that induces binocular rivalry with bistable perception that alternates between the two images. The relationship between binocular integration and interocular suppression concerns how our brain processes binocular inputs to form unified visual awareness but remains unclear. Here, a series of psychophysical experiments were conducted to address this question, revealing that these collaborative and competitive binocular interactions are interconnected and would mediate one another according to their strength. Specifically, Experiments 1a and 1b showed that the presence of binocular rivalry inhibited peripheral stereopsis, significantly elevating the stereo threshold, with higher elevation resulting from increasing rivalry contrast. Experiments 2a and 2b showed that existing stereopsis with increasing binocular disparity balanced the dynamics of peripheral binocular rivalry, rendering more equivalent eye dominance. Based on these interactions, we suggest that binocular integration and interocular suppression may mediate one another through an overlapping mechanism for regulating eye dominance, with strong stereo percepts tending to reduce eye dominance and strong rivalry tending to increase eye dominance.

Perceiving depth from texture and disparity cues: Evidence for a non-probabilistic account of cue integration.

Bayesian inference theories have been extensively used to model how the brain derives three-dimensional (3D) information from ambiguous visual input. In particular, the maximum likelihood estimation (MLE) model combines estimates from multiple depth cues according to their relative reliability to produce the most probable 3D interpretation. Here, we tested an alternative theory of cue integration, termed the intrinsic constraint (IC) theory, which postulates that the visual system derives the most stable, not most probable, interpretation of the visual input amid variations in viewing conditions. The vector sum model provides a normative approach for achieving this goal where individual cue estimates are components of a multidimensional vector whose norm determines the combined estimate. Individual cue estimates are not accurate but related to distal 3D properties through a deterministic mapping. In three experiments, we show that the IC theory can more adeptly account for 3D cue integration than MLE models. In Experiment 1, we show systematic biases in the perception of depth from texture and depth from binocular disparity. Critically, we demonstrate that the vector sum model predicts an increase in perceived depth when these cues are combined. In Experiment 2, we illustrate the IC theory radical reinterpretation of the just noticeable difference (JND) and test the related vector sum model prediction of the classic finding of smaller JNDs for combined-cue versus single-cue stimuli. In Experiment 3, we confirm the vector sum prediction that biases found in cue integration experiments cannot be attributed to flatness cues, as the MLE model predicts.

Topical Review: Methodological Variables in Clinical and Laboratory Measurements of Fixation Disparity.

SIGNIFICANCE: Fixation disparity is a small vergence error that does not disrupt fusion. Fixation disparity measures correlate with binocular symptoms. This article covers methodological differences between clinical fixation disparity measurement devices, findings when objective and subjective fixation disparities are compared, and the potential impact of binocular capture on fixation disparity measurements. Fixation disparity is a small vergence error that occurs in nonstrabismic individuals and does not disrupt fusion. This article reviews clinical fixation disparity variables and their clinical diagnostic value. Clinical devices that are used to measure these variables are described, as are studies in which the output from these devices has been compared. Methodological differences between the devices such as the location of the fusional stimulus, the rate at which judgments of dichoptic alignment are made, and the strength of the accommodative stimulus are all considered. In addition, the article covers theories of the neural origins of fixation disparity and control system models incorporating fixation disparity. Studies in which objective fixation disparities (oculomotor portion of fixation disparity assessed with an eye tracker) and subjective fixation disparities (sensory portion of fixation disparity assessed psychophysically with dichoptic Nonius lines) have been compared are also examined, and consideration is given to why some investigators find differences in these measures, whereas other investigators do not. The conclusion thus far is that there are likely complex interactions between vergence adaptation, accommodation, and the location of the fusional stimulus that lead to differences in objective and subjective fixation disparity measures. Finally, capture of the visual direction of monocular stimuli by adjacent fusional stimuli and the implications for fixation disparity measures are considered.

Subtractive adaptation is a more effective and general mechanism in binocular rivalry than divisive adaptation.

The activity of neurons is influenced by random fluctuations and can be strongly modulated by firing rate adaptation, particularly in sensory systems. Still, there is ongoing debate about the characteristics of neuronal noise and the mechanisms of adaptation, and even less is known about how exactly they affect perception. Noise and adaptation are critical in binocular rivalry, a visual phenomenon where two images compete for perceptual dominance. Here, we investigated the effects of different noise processes and adaptation mechanisms on visual perception by simulating a model of binocular rivalry with Gaussian white noise, Ornstein-Uhlenbeck noise, and pink noise, in variants with divisive adaptation, subtractive adaptation, and without adaptation. By simulating the nine models in parameter space, we find that white noise only produces rivalry when paired with subtractive adaptation and that subtractive adaptation reduces the influence of noise intensity on rivalry strength and introduces convergence of the mean percept duration, an important metric of binocular rivalry, across all noise processes. In sum, our results show that white noise is an insufficient description of background activity in the brain and that subtractive adaptation is a stronger and more general switching mechanism in binocular rivalry than divisive adaptation, with important noise-filtering properties.

Double fusion, a depth perception mechanism in Panum's limiting case.

The origin of depth in Panum's limiting case is unclear at present, so we investigated the depth perception mechanism using a triangle type of Panum's stimulus with a slant effect and clear criterion. Experiment 1 explored whether participants can correctly perceive fixation and nonfixation features using the fixation point and quick representation of stimuli, then examined whether participants' depth judgments supported double fusion or single fusion. The results of Experiment 1 showed that participants could correctly perceive the depth of fixation and nonfixation features. That is, it supported double fusion. In Experiment 2, we examined whether the depth perceived by observers comes from depth contrast. The results of Experiment 2 showed that the depth of the two features perceived after binocular fusion did not originate from the depth contrast. The findings suggest that the depth perception mechanism of Panum's limiting case is more likely to be double fusion.

Detection of vertical interocular phase disparities using luster as cue.

Interocular disparities in contrast generate an impression of binocular luster, providing a cue for their detection. Disparities in the carrier spatial phase of horizontally oriented Gabor patches also generate an impression of luster, so the question arises as to whether it is the disparities in local contrast that accompany the phase disparities that give rise to the luster. We examined this idea by comparing the detection of interocular spatial phase disparities with that of interocular contrast disparities in Gabor patches, in the latter case that differed in overall contrast rather than phase between the eyes. When bandwidth was held constant and Gabor spatial frequency was varied, the detection of phase and contrast disparities followed a similar pattern. However, when spatial frequency was fixed and Gabor envelope standard deviation (and hence number of modulation cycles) was varied, thresholds for detecting phase disparities followed a U-shaped function of Gabor standard deviation, whereas thresholds for contrast disparities, following an initial decline, were more-or-less constant as a function of Gabor standard deviation. After reviewing a number of possible explanations for the U-shape found with phase disparities, we suggest that the likely cause is binocular sensory fusion, the strength of which increases with the number of modulation cycles. Binocular sensory fusion would operate to reduce phase but not contrast disparities, thus selectively elevating phase disparity thresholds.