Color and gloss constancy under diverse lighting environments.

When we look at an object, we simultaneously see how glossy or matte it is, how light or dark, and what color. Yet, at each point on the object's surface, both diffuse and specular reflections are mixed in different proportions, resulting in substantial spatial chromatic and luminance variations. To further complicate matters, this pattern changes radically when the object is viewed under different lighting conditions. The purpose of this study was to simultaneously measure our ability to judge color and gloss using an image set capturing diverse object and illuminant properties. Participants adjusted the hue, lightness, chroma, and specular reflectance of a reference object so that it appeared to be made of the same material as a test object. Critically, the two objects were presented under different lighting environments. We found that hue matches were highly accurate, except for under a chromatically atypical illuminant. Chroma and lightness constancy were generally poor, but these failures correlated well with simple image statistics. Gloss constancy was particularly poor, and these failures were only partially explained by reflection contrast. Importantly, across all measures, participants were highly consistent with one another in their deviations from constancy. Although color and gloss constancy hold well in simple conditions, the variety of lighting and shape in the real world presents significant challenges to our visual system's ability to judge intrinsic material properties.

Bright-light distractions and visual performance.

Visual distractions pose a significant risk to transportation safety, with laser attacks against aircraft pilots being a common example. This study used a research-grade High Dynamic Range (HDR) display to produce bright-light distractions for 12 volunteer participants performing a combined visual task across central and peripheral visual fields. The visual scene had an average luminance of 10 cd∙m-2 with targets of approximately 0.5° angular size, while the distractions had a maximum luminance of 9,000 cd∙m-2 and were 3.6° in size. The dependent variables were the mean fixation duration during task execution (representative of information processing time), and the critical stimulus duration required to support a target level of performance (representative of task efficiency). The experiment found a statistically significant increase in mean fixation duration, rising from 192 ms without distractions to 205 ms with bright-light distractions (p = 0.023). This indicates a decrease in visibility of the low contrast targets or an increase in cognitive workload that required greater processing time for each fixation in the presence of the bright-light distractions. Mean critical stimulus duration was not significantly affected by the distraction conditions used in this study. Future experiments are suggested to replicate driving and/or piloting tasks and employ bright-light distractions based on real-world data, and we advocate the use of eye-tracking metrics as sensitive measures of changes in performance.

Task-dependent extraction of information from videos of iridescent and glossy samples.

We present an exploratory study on iridescence that revealed systematic differences in the perceptual clustering of glossy and iridescent samples that was driven by instructions to focus on either the material or the color properties of the samples. Participants' similarity ratings of pairs of video stimuli, showing the samples from multiple views, were analyzed using multidimensional scaling (MDS), and differences between the MDS solutions for the two tasks were consistent with flexible weighting of information from different views of the samples. These findings point to ecological implications for how viewers perceive and interact with the color-changing properties of iridescent objects.

Modeling surface color discrimination under different lighting environments using image chromatic statistics and convolutional neural networks.

We modeled discrimination thresholds for object colors under different lighting environments [J. Opt. Soc. Am. 35, B244 (2018)]. First, we built models based on chromatic statistics, testing 60 models in total. Second, we trained convolutional neural networks (CNNs), using 160,280 images labeled by either the ground-truth or human responses. No single chromatic statistics model was sufficient to describe human discrimination thresholds across conditions, while human-response-trained CNNs nearly perfectly predicted human thresholds. Guided by region-of-interest analysis of the network, we modified the chromatic statistics models to use only the lower regions of the objects, which substantially improved performance.

How do (perceptual) distracters distract?

When a target stimulus occurs in the presence of distracters, decisions are less accurate. But how exactly do distracters affect choices? Here, we explored this question using measurement of human behaviour, psychophysical reverse correlation and computational modelling. We contrasted two models: one in which targets and distracters had independent influence on choices (independent model) and one in which distracters modulated choices in a way that depended on their similarity to the target (interaction model). Across three experiments, participants were asked to make fine orientation judgments about the tilt of a target grating presented adjacent to an irrelevant distracter. We found strong evidence for the interaction model, in that decisions were more sensitive when target and distracter were consistent relative to when they were inconsistent. This consistency bias occurred in the frame of reference of the decision, that is, it operated on decision values rather than on sensory signals, and surprisingly, it was independent of spatial attention. A normalization framework, where target features are normalized by the expectation and variability of the local context, successfully captures the observed pattern of results.

Low level visual features support robust material perception in the judgement of metallicity.

The human visual system is able to rapidly and accurately infer the material properties of objects and surfaces in the world. Yet an inverse optics approach-estimating the bi-directional reflectance distribution function of a surface, given its geometry and environment, and relating this to the optical properties of materials-is both intractable and computationally unaffordable. Rather, previous studies have found that the visual system may exploit low-level spatio-chromatic statistics as heuristics for material judgment. Here, we present results from psychophysics and modeling that supports the use of image statistics heuristics in the judgement of metallicity-the quality of appearance that suggests an object is made from metal. Using computer graphics, we generated stimuli that varied along two physical dimensions: the smoothness of a metal object, and the evenness of its transparent coating. This allowed for the exploration of low-level image statistics, whilst ensuring that each stimulus was a naturalistic, physically plausible image. A conjoint-measurement task decoupled the contributions of these dimensions to the perception of metallicity. Low-level image features, as represented in the activations of oriented linear filters at different spatial scales, were found to correlate with the dimensions of the stimulus space, and decision-making models using these activations replicated observer performance in perceiving differences in metal smoothness and coating bumpiness, and judging metallicity. Importantly, the performance of these models did not deteriorate when objects were rotated within their simulated scene, with corresponding changes in image properties. We therefore conclude that low-level image features may provide reliable cues for the robust perception of metallicity.

Emulated retinal image capture (ERICA) to test, train and validate processing of retinal images.

High resolution retinal imaging systems, such as adaptive optics scanning laser ophthalmoscopes (AOSLO), are increasingly being used for clinical research and fundamental studies in neuroscience. These systems offer unprecedented spatial and temporal resolution of retinal structures in vivo. However, a major challenge is the development of robust and automated methods for processing and analysing these images. We present ERICA (Emulated Retinal Image CApture), a simulation tool that generates realistic synthetic images of the human cone mosaic, mimicking images that would be captured by an AOSLO, with specified image quality and with corresponding ground-truth data. The simulation includes a self-organising mosaic of photoreceptors, the eye movements an observer might make during image capture, and data capture through a real system incorporating diffraction, residual optical aberrations and noise. The retinal photoreceptor mosaics generated by ERICA have a similar packing geometry to human retina, as determined by expert labelling of AOSLO images of real eyes. In the current implementation ERICA outputs convincingly realistic en face images of the cone photoreceptor mosaic but extensions to other imaging modalities and structures are also discussed. These images and associated ground-truth data can be used to develop, test and validate image processing and analysis algorithms or to train and validate machine learning approaches. The use of synthetic images has the advantage that neither access to an imaging system, nor to human participants is necessary for development.

A Modeling Study of the Emergence of Eye Position Gain Fields Modulating the Responses of Visual Neurons in the Brain.

The responses of many cortical neurons to visual stimuli are modulated by the position of the eye. This form of gain modulation by eye position does not change the retinotopic selectivity of the responses, but only changes the amplitude of the responses. Particularly in the case of cortical responses, this form of eye position gain modulation has been observed to be multiplicative. Multiplicative gain modulated responses are crucial to encode information that is relevant to high-level visual functions, such as stable spatial awareness, eye movement planning, visual-motor behaviors, and coordinate transformation. Here we first present a hardwired model of different functional forms of gain modulation, including peaked and monotonic modulation by eye position. We use a biologically realistic Gaussian function to model the influence of the position of the eye on the internal activation of visual neurons. Next we show how different functional forms of gain modulation by eye position may develop in a self-organizing neural network model of visual neurons. A further contribution of our work is the investigation of the influence of the width of the eye position tuning curve on the development of a variety of forms of eye position gain modulation. Our simulation results show how the width of the eye position tuning curve affects the development of different forms of gain modulation of visual responses by the position of the eye.

Demonstrating a multi-primary high dynamic range display system for vision experiments.

We describe the design, construction, calibration, and characterization of a multi-primary high dynamic range (MPHDR) display system for use in vision research. The MPHDR display is the first system to our knowledge to allowfor spatially controllable, high dynamic range stimulus generation using multiple primaries.We demonstrate the high luminance, high dynamic range, and wide color gamut output of the MPHDR display. During characterization, the MPHDR display achieved a maximum luminance of 3200 cd=m2, a maximum contrast range of 3; 240; 000 V 1, and an expanded color gamut tailored to dedicated vision research tasks that spans beyond traditional sRGB displays. We discuss how the MPHDR display could be optimized for psychophysical experiments with photoreceptor isolating stimuli achieved through the method of silent substitution. We present an example case of a range of metameric pairs of melanopsin isolating stimuli across different luminance levels, from an available melanopsin contrast of117%at 75 cd=m2 to a melanopsin contrast of23%at 2000 cd=m2.

The impact on emotion classification performance and gaze behavior of foveal versus extrafoveal processing of facial features.

At normal interpersonal distances all features of a face cannot fall within one's fovea simultaneously. Given that certain facial features are differentially informative of different emotions, does the ability to identify facially expressed emotions vary according to the feature fixated and do saccades preferentially seek diagnostic features? Previous findings are equivocal. We presented faces for a brief time, insufficient for a saccade, at a spatial position that guaranteed that a given feature-an eye, cheek, the central brow, or mouth-fell at the fovea. Across 2 experiments, observers were more accurate and faster at discriminating angry expressions when the high spatial-frequency information of the brow was projected to their fovea than when 1 or other cheek or eye was. Performance in classifying fear and happiness (Experiment 1) was not influenced by whether the most informative features (eyes and mouth, respectively) were projected foveally or extrafoveally. Observers more accurately distinguished between fearful and surprised expressions (Experiment 2) when the mouth was projected to the fovea. Reflexive first saccades tended toward the left and center of the face rather than preferentially targeting emotion-distinguishing features. These results reflect the integration of task-relevant information across the face constrained by the differences between foveal and extrafoveal processing (Peterson & Eckstein, 2012). (PsycINFO Database Record (c) 2020 APA, all rights reserved).

What makes a microsaccade? A review of 70 years of research prompts a new detection method.

A new method for detecting microsaccades in eye-movement data is presented, following a review of reported microsaccade properties between the 1940s and today. The review focuses on the parameter ranges within which certain physical markers of microsaccades are thought to occur, as well as any features of microsaccades that have been stably reported over time. One feature of microsaccades, their binocularity, drives the new microsaccade detection method. The binocular correlation method for microsaccade detection is validated on two datasets of binocular eye-movements recorded using video-based systems: one collected as part of this study, and one from Nyström et al, 2017. Comparisons between detection methods are made using precision-recall statistics. This confirms that the binocular correlation method performs well when compared to manual coders and performs favourably compared to the commonly used Engbert & Kliegl (2003) method with subsequent modifications (Engbert & Mergenthaler, 2006). The binocular correlation microsaccade detection method is easy to implement and MATLAB code is made available to download.

Hyperspectral environmental illumination maps: characterizing directional spectral variation in natural environments.

Objects placed in real-world scenes receive incident light from every direction, and the spectral content of this light may vary from one direction to another. In computer graphics, environmental illumination is approximated using maps that specify illumination at a point as a function of incident angle. However, to-date, existing public databases of environmental illumination specify only three colour channels (RGB). We have captured a new set of 12 environmental illumination maps (eight outdoor scenes; four indoor scenes) using a hyperspectral imaging system with 33 spectral channels. The data reveal a striking directional variation of spectral distribution of lighting in natural environments. We discuss limitations of using daylight models to describe natural environmental illumination.

Hand-Foot Coupling: An Advantage for Crossed Legs.

It is difficult to perform distinct, simultaneous motor actions with the ipsilateral hand and foot; for example, clockwise circles with the right hand and counter-clockwise circles with the right foot. By chance, we discovered that this hand-foot coupling task is easier when seated with legs crossed. We consider various explanations. First, that there are reduced demands on the contralateral hemisphere when the motor programme of the right foot is executed on the left side of the body. Second, that the legs-crossed scenario is easier because movements are symmetrical with respect to body midline. By considering related motor actions, we conclude that neither of these explanations provides a full account. Thus, we suggest a third explanation, which is that coupling effects are reduced by virtue of increased postural stability and reduced anticipatory postural adjustments.

Self-organising coordinate transformation with peaked and monotonic gain modulation in the primate dorsal visual pathway.

We study a self-organising neural network model of how visual representations in the primate dorsal visual pathway are transformed from an eye-centred to head-centred frame of reference. The model has previously been shown to robustly develop head-centred output neurons with a standard trace learning rule, but only under limited conditions. Specifically it fails when incorporating visual input neurons with monotonic gain modulation by eye-position. Since eye-centred neurons with monotonic gain modulation are so common in the dorsal visual pathway, it is an important challenge to show how efferent synaptic connections from these neurons may self-organise to produce head-centred responses in a subpopulation of postsynaptic neurons. We show for the first time how a variety of modified, yet still biologically plausible, versions of the standard trace learning rule enable the model to perform a coordinate transformation from eye-centred to head-centred reference frames when the visual input neurons have monotonic gain modulation by eye-position.

Beyond scattering and absorption: Perceptual unmixing of translucent liquids.

Is perception of translucence based on estimations of scattering and absorption of light or on statistical pseudocues associated with familiar materials? We compared perceptual performance with real and computer-generated stimuli. Real stimuli were glasses of milky tea. Milk predominantly scatters light and tea absorbs it, but since the tea absorbs less as the milk concentration increases, the effects of milkiness and strength on scattering and absorption are not independent. Conversely, computer-generated stimuli were glasses of "milky tea" in which absorption and scattering were independently manipulated. Observers judged tea concentrations regardless of milk concentrations, or vice versa. Maximum-likelihood conjoint measurement was used to estimate the contributions of each physical component-concentrations of milk and tea, or amounts of scattering and absorption-to perceived milkiness or tea strength. Separability of the two physical dimensions was better for real than for computer-generated teas, suggesting that interactions between scattering and absorption were correctly accounted for in perceptual unmixing, but unmixing was always imperfect. Since the real and rendered stimuli represent different physical processes and therefore differ in their image statistics, perceptual judgments with these stimuli allowed us to identify particular pseudocues (presumably learned with real stimuli) that explain judgments with both stimulus sets.

Raw high-speed schlieren footage of acoustic waves in air for subsequent computational analysis and audio recovery.

The dataset presented here comprises 79 raw, unprocessed video files obtained from the high-speed schlieren imaging of acoustic waves in air, each with a corresponding metadata file. The majority of the footage was recorded at 50,000 frames per second, with each condition filmed for a duration of 20 ms. The dataset includes footage corresponding to a wide range of signals, in terms of waveform, frequency, and amplitude, as well as varied imaging parameters (exposure, frame rate, spatial resolution). This Data in Brief article is to accompany the research article "Visualization of acoustic waves in air and subsequent audio recovery with a high-speed schlieren imaging system: Experimental and computational development of a schlieren microphone" [1].

Delayed S-cone sensitivity losses following the onset of intense yellow backgrounds linked to the lifetime of a photobleaching product?

Thirty years ago, Mollon, Stockman, & Polden (1987) reported that after the onset of intense yellow 581-nm backgrounds, S-cone threshold rose unexpectedly for several seconds before recovering to the light-adapted steady-state value-an effect they called: "transient-tritanopia of the second kind" (TT2). Given that 581-nm lights have little direct effect on S-cones, TT2 must arise indirectly from the backgrounds' effects on the L- and M-cones. We attribute the phenomenon to the action of an unknown L- and M-cone photobleaching product, X, which acts at their outputs like an "equivalent" background light that then inhibits S-cones at a cone-opponent, second-site. The time-course of TT2 is similar in form to the lifetime of X in a two-stage, first-order biochemical reaction A→X→C with successive best-fitting time-constants of 3.09 ± 0.35 and 7.73 ± 0.70 s. Alternatively, with an additional slowly recovering exponential "restoring-force" with a best-fitting time-constant 23.94 ± 1.42 s, the two-stage best-fitting time-constants become 4.15 ± 0.62 and 6.79 ± 1.00 s. Because the time-constants are roughly independent of the background illumination, and thus the rate of photoisomerization, A→X is likely to be a reaction subsidiary to the retinoid cycle, perhaps acting as a buffer when the bleaching rate is too high. X seems to be logarithmically related to S-cone threshold, which may result from the logarithmic cone-opponent, second-site response compression after multiplicative first-site adaptation. The restoring-force may be the same cone-opponent force that sets the rate of S-cone recovery following the unusual threshold increase following the offset of dimmer yellow backgrounds, an effect known as "transient-tritanopia" (TT1).

Are hue and saturation carried in different neural channels?

Chromatic discrimination data show that a smaller physical stimulus change is required to detect a change in hue than to detect a change in saturation [Palette30, 21 (1968); Proc. R. Soc. London Ser. B283, 20160164 (2016)PRLBA40080-464910.1098/rspb.2016.0164], and, on this basis, it has been suggested that hue and saturation are carried in different neural channels [Color Space and Its Divisions: Color Order from Antiquity to the Present (Wiley, 2003), p. 311]. We used an adaptation paradigm to test explicitly for separate mechanisms, measuring hue and saturation detection thresholds before and after adaptation to hue and saturation stimuli. Within-condition adaptation did not elevate detection thresholds significantly more than between-condition adaptation. We therefore did not find psychophysical evidence for a neural channel that extracts hue thresholds more effectively than the neural channel or channels that determine saturation thresholds.

Discrimination of spectral reflectance under environmental illumination.

Color constancy is the ability to recover a stable perceptual estimate of surface reflectance, regardless of the lighting environment. However, we know little about how observers make judgments of the surface color of glossy objects, particularly in complex lighting environments that introduce complex spatial patterns of chromatic variation across an object's surface. To address this question, we measured thresholds for reflectance discrimination using computer-rendered stimuli under environmental illumination. In Experiment 1, we found that glossiness and shape had small effects on discrimination thresholds. Importantly, discrimination ellipses extended along the direction in which the chromaticities in the environmental illumination spread. In Experiment 2, we also found that the observers' abilities to judge surface colors were worse in lighting environments with an atypical chromatic distribution.

Compact, modular and in-plane AOSLO for high-resolution retinal imaging.

The adaptive optics scanning laser ophthalmoscope (AOSLO) was first developed in 2002 and since then the technology has been adopted in several laboratories around the world, for both clinical and psychophysical research. There have been a few major design implementations of the AOSLO. The first used on-axis tilted spherical mirrors in a planar arrangement, and the second minimized the build up of astigmatism present in the first design by using a non-planar arrangement. Other designs have avoided astigmatism by using custom-made toroidal mirrors or by using lenses on-axis, rather than mirrors. We present a new design implementation for an AOSLO that maintains a planar optical alignment without the build up astigmatism using compact, reconfigurable modules based on an Offner relay system. We additionally use an off-the-shelf digital oscilloscope for data capture and custom-written Python code for generating and analyzing the retinal images. This design results in a compact system that is simple to align and, being composed of modular relays, has the potential for additional components to be added. We show that this system maintains diffraction-limited image quality across the field of view and that cones are clearly resolved in the central retina. The modular relay design is generally applicable to any system requiring one or more components in the pupil conjugate plane. This is likely to be useful for any point-scanned system, such as a standard scanning laser ophthalmoscope or non-ophthalmic confocal imaging system.