Perceptual task drives later fixations and long latency saccades, while early fixations and short latency saccades are more automatic.

We used a simple stimulus, dissociating perceptually relevant information in space, to differentiate between bottom-up and task-driven fixations. Six participants viewed a dynamic scene showing the reaction of an elastic object fixed to the ceiling being hit. In one condition they had to judge the object's stiffness and in the other condition its lightness. The results show that initial fixations tend to land in the centre of an object, independent of the task. After the initial fixation, participants tended to look at task diagnostic regions. This fixation behaviour correlates with high perceptual performance. Similarly, low-latency saccades lead to fixations that do not depend on the task, whereas higher latency does.

Color constancy mechanisms in virtual reality environments.

Prior research has demonstrated high levels of color constancy in real-world scenarios featuring single light sources, extensive fields of view, and prolonged adaptation periods. However, exploring the specific cues humans rely on becomes challenging, if not unfeasible, with actual objects and lighting conditions. To circumvent these obstacles, we employed virtual reality technology to craft immersive, realistic settings that can be manipulated in real time. We designed forest and office scenes illuminated by five colors. Participants selected a test object most resembling a previously shown achromatic reference. To study color constancy mechanisms, we modified scenes to neutralize three contributors: local surround (placing a uniform-colored leaf under test objects), maximum flux (keeping the brightest object constant), and spatial mean (maintaining a neutral average light reflectance), employing two methods for the latter: changing object reflectances or introducing new elements. We found that color constancy was high in conditions with all cues present, aligning with past research. However, removing individual cues led to varied impacts on constancy. Local surrounds significantly reduced performance, especially under green illumination, showing strong interaction between greenish light and rose-colored contexts. In contrast, the maximum flux mechanism barely affected performance, challenging assumptions used in white balancing algorithms. The spatial mean experiment showed disparate effects: Adding objects slightly impacted performance, while changing reflectances nearly eliminated constancy, suggesting human color constancy relies more on scene interpretation than pixel-based calculations.

Context effects on the perception of saturation of fruit colors in still-life paintings.

Still-life painters, especially of the so-called Golden Age (17th century) in the Netherlands, are famous for their masterful techniques of rendering reality. Their amazing abilities to depict different material properties of fruits and flowers are stunning. But how important are these careful arrangements of different objects for the perception of an individual item? Is the perceived color saturation of a single fruit influenced by its surrounding context? We selected fruits in still-life paintings as stimuli to investigate whether and how perceived saturations of fruits were affected by their original contexts. In our study, we focused especially on effects of five context properties: complementary colors, chromatic and luminance contrast, object overlap, and surround variance. Six fruit varieties depicted in high-quality digital reproductions of 48 classic and eight varieties in 64 more recent, modern still-life paintings were selected. In a single trial, eight images of fruits of the same variety appeared on a neutral gray background; half were single fruit cutouts, and the other half were the same fruits embedded in their circular contexts. Fifteen participants ranked all eight images according to perceived color saturations of the fruits. Saturation ratings showed a high agreement of 77%. Surrounding contexts led to an increase in perceived saturation of central fruits. This effect was mainly driven by object overlap, the presence of the central fruit type also in the context, and surround variance. Chroma contrast between fruits and contexts decreased saturation significantly. No significant context effects were found for complementary colors or luminance contrast. Our results show that in paintings, many of the cues that are usually experimentally isolated occur in interesting combinations and lead to an increase in perceived saturation that makes fruit objects more appealing and convincing.

Perception of saturation in natural objects.

The distribution of colors across a surface depends on the interaction between its surface properties, its shape, and the lighting environment. Shading, chroma, and lightness are positively correlated: points on the object that have high luminance also have high chroma. Saturation, typically defined as the ratio of chroma to lightness, is therefore relatively constant across an object. Here we explored to what extent this relationship affects perceived saturation of an object. Using images of hyperspectral fruit and rendered matte objects, we manipulated the lightness-chroma correlation (positive or negative) and asked observers which of two objects appeared more saturated. Despite the negative-correlation stimulus having greater mean and maximum chroma, lightness, and saturation than the positive, observers overwhelmingly chose the positive as more saturated. This suggests that simple colorimetric statistics do not accurately represent perceived saturation of objects-observers likely base their judgments on interpretations about the cause of the color distribution.

Foveal to peripheral extrapolation of facial emotion.

Peripheral vision is characterized by poor resolution. Recent evidence from brightness perception suggests that missing information is filled out with information at fixation. Here we show a novel filling-out mechanism: when participants are presented with a crowd of faces, the perceived emotion of faces in peripheral vision is biased towards the emotion of the face at fixation. This mechanism is particularly important in social situations where people often need to perceive the overall mood of a crowd. Some faces in the crowd are more likely to catch people's attention and be looked at directly, while others are only seen peripherally. Our findings suggest that the perceived emotion of these peripheral faces, and the overall perceived mood of the crowd, is biased by the emotions of the faces that people look at directly.

Unsupervised learning of haptic material properties.

When touching the surface of an object, its spatial structure translates into a vibration on the skin. The perceptual system evolved to translate this pattern into a representation that allows to distinguish between different materials. Here, we show that perceptual haptic representation of materials emerges from efficient encoding of vibratory patterns elicited by the interaction with materials. We trained a deep neural network with unsupervised learning (Autoencoder) to reconstruct vibratory patterns elicited by human haptic exploration of different materials. The learned compressed representation (i.e., latent space) allows for classification of material categories (i.e., plastic, stone, wood, fabric, leather/wool, paper, and metal). More importantly, classification performance is higher with perceptual category labels as compared to ground truth ones, and distances between categories in the latent space resemble perceptual distances, suggesting a similar coding. Crucially, the classification performance and the similarity between the perceptual and the latent space decrease with decreasing compression level. We could further show that the temporal tuning of the emergent latent dimensions is similar to properties of human tactile receptors.

Colour Calibration of a Head Mounted Display for Colour Vision Research Using Virtual Reality.

Virtual reality (VR) technology offers vision researchers the opportunity to conduct immersive studies in simulated real-world scenes. However, an accurate colour calibration of the VR head mounted display (HMD), both in terms of luminance and chromaticity, is required to precisely control the presented stimuli. Such a calibration presents significant new challenges, for example, due to the large field of view of the HMD, or the software implementation used for scene rendering, which might alter the colour appearance of objects. Here, we propose a framework for calibrating an HMD using an imaging colorimeter, the I29 (Radiant Vision Systems, Redmond, WA, USA). We examine two scenarios, both with and without using a rendering software for visualisation. In addition, we present a colour constancy experiment design for VR through a gaming engine software, Unreal Engine 4. The colours of the objects of study are chosen according to the previously defined calibration. Results show a high-colour constancy performance among participants, in agreement with recent studies performed on real-world scenarios. Our studies show that our methodology allows us to control and measure the colours presented in the HMD, effectively enabling the use of VR technology for colour vision research.

Underconfidence in peripheral vision.

Our visual experience appears uniform across the visual field, despite the poor resolution of peripheral vision. This may be because we do not notice that we are missing details in the periphery of our visual field and believe that peripheral vision is just as rich as central vision. In other words, the uniformity of the visual scene could be explained by a metacognitive bias. We deployed a confidence forced-choice method to measure metacognitive performance in peripheral as compared to central vision. Participants judged the orientation of gratings presented in central and peripheral vision, and reported whether they thought they were more likely to be correct in the perceptual decision for the central or for the peripheral stimulus. Observers were underconfident in the periphery: higher sensory evidence in the periphery was needed to equate confidence choices between central and peripheral perceptual decisions. When performance on the central and peripheral tasks was matched, observers were still more confident in their ability to report the orientation of the central gratings over the one of the peripheral gratings. In a second experiment, we measured metacognitive sensitivity, as the difference in perceptual sensitivity between perceptual decisions that are chosen with high confidence and decisions that are chosen with low confidence. Results showed that metacognitive sensitivity is lower when participants compare central to peripheral perceptual decisions compared to when they compare peripheral to peripheral or central to central perceptual decisions. In a third experiment, we showed that peripheral underconfidence does not arise because observers based confidence judgments on stimulus size or contrast range rather than on perceptual performance. Taken together, results indicate that humans are impaired in comparing central with peripheral perceptual performance, but metacognitive biases cannot explain our impression of uniformity, as this would require peripheral overconfidence.

Target Search and Inspection Strategies in Haptic Search.

Haptic search is a common everyday task, usually consisting of two processes: target search and target analysis. During target search we need to know where our fingers are in space, remember the already completed path and the outline of the remaining space. During target analysis we need to understand whether the detected potential target is the desired one. Here we characterized dynamics of exploratory movements in these two processes. In our experiments participants searched for a particular configuration of symbols on a rectangular tactile display. We observed that participants preferentially moved the hand parallel to the edges of the tactile display during target search, which possibly eased orientation within the search space. After a potential target was detected by any of the fingers, there was higher probability that subsequent exploration was performed by the index or the middle finger. At the same time, these fingers dramatically slowed down. Being in contact with the potential target, the index and the middle finger moved within a smaller area than the other fingers, which rather seemed to move away to leave them space. These results suggest that the middle and the index finger are specialized for fine analysis in haptic search.

Surface properties and the perception of color.

We examined whether perception of color saturation and lightness depends on the three-dimensional (3D) shape and surface gloss of surfaces rendered to have different hues. In Experiment 1, we parametrically varied specular roughness of predominantly planar surfaces with different mesoscopic relief heights. The orientation of surfaces was varied relative to the light source and observer. Observers matched perceived lightness and chroma (effectively saturation) using spherical objects rendered using CIE LCH color space. We observed strong interactions between perceived saturation and lightness with changes in surface orientation and surface properties (specular roughness and 3D relief height). Declines in saturation and increases in lightness were observed with increasing specular roughness. Changes in relief height had greater effects on perceived saturation and lightness for blue hues compared with reddish and greenish hues. Experiment 2 found inverse correlations between perceived gloss and specular roughness across conditions. Experiment 3 estimated perceived specular coverage and found that a weighted combination of perceived gloss and specular coverage could account for perceived color saturation and lightness, with different coefficients accounting for the perceptual experience for each of the three hue conditions. These findings suggest that perceived color saturation and lightness depend on the separation of specular highlights from diffuse shading informative of chromatic surface reflectance.

Deep neural network model of haptic saliency.

Haptic exploration usually involves stereotypical systematic movements that are adapted to the task. Here we tested whether exploration movements are also driven by physical stimulus features. We designed haptic stimuli, whose surface relief varied locally in spatial frequency, height, orientation, and anisotropy. In Experiment 1, participants subsequently explored two stimuli in order to decide whether they were same or different. We trained a variational autoencoder to predict the spatial distribution of touch duration from the surface relief of the haptic stimuli. The model successfully predicted where participants touched the stimuli. It could also predict participants' touch distribution from the stimulus' surface relief when tested with two new groups of participants, who performed a different task (Exp. 2) or explored different stimuli (Exp. 3). We further generated a large number of virtual surface reliefs (uniformly expressing a certain combination of features) and correlated the model's responses with stimulus properties to understand the model's preferences in order to infer which stimulus features were preferentially touched by participants. Our results indicate that haptic exploratory behavior is to some extent driven by the physical features of the stimuli, with e.g. edge-like structures, vertical and horizontal patterns, and rough regions being explored in more detail.

How to make a #theDress.

If we completely understand how a phenomenon works, we should be able to produce it ourselves. However, the individual differences in color appearance observed with #theDress seem to be a peculiarity of that photo, and it remains unclear how the proposed mechanisms underlying #theDress can be generalized to other images. Here, we developed a simple algorithm that transforms any image with bicolored objects into an image with the properties of #theDress. We measured the colors perceived in such images and compared them to those perceived in #theDress. Color adjustments confirmed that observers strongly differ in how they perceive the colors of the new images in a similar way as for #theDress. Most importantly, these differences were not unsystematic, but correlated with how observers perceive #theDress. These results imply that the color distribution is sufficient to produce the striking individual differences in color perception originally observed with #theDress-at least as long as the image appears realistic and hence compels the viewer to make assumptions about illuminations and surfaces. The algorithm can be used for stimulus production beyond this study.

Color consistency in the appearance of bleached fabrics.

Human observers are remarkably good at perceiving constant object color across illumination changes. However, there are numerous other factors that can modulate surface appearance, such as aging, bleaching, staining, or soaking. Despite this, we are often able to identify material properties across such transformations. Little is known about how and to what extent we can compensate for the accompanying color transformations. Here we investigated whether humans could reproduce the original color of bleached fabrics. We treated 12 different fabric samples with a commercial bleaching product. Bleaching increased luminance and decreased saturation. We presented photographs of the original and bleached samples on a computer screen and asked observers to match the fabric colors to an adjustable matching disk. Different groups of observers produced matches for original and bleached samples. One group of observers were instructed to match the color of the bleached samples as they were before bleaching (i.e., compensate for the effects of bleaching); another, to accurately match color appearance. Observers did compensate significantly for the effects of bleaching when instructed to do so, but not in the appearance match condition. Results of a second experiment suggest that observers achieve color consistency, at least in part, through a strategy based on local spatial differences within the bleached samples. According to the results of a third experiment, these local spatial differences are likely to be the perceptual image cues that allow participants to determine whether a sample is bleached. When the effect of bleaching was limited or uniformly distributed across a sample's surface, observers were uncertain about the bleaching magnitude and seemed to apply cognitive strategies to achieve color consistency.

Perceptually Validated Cross-Renderer Analytical BRDF Parameter Remapping.

Material appearance of rendered objects depends on the underlying BRDF implementation used by rendering software packages. A lack of standards to exchange material parameters and data (between tools) means that artists in digital 3D prototyping and design, manually match the appearance of materials to a reference image. Since their effect on rendered output is often non-uniform and counter intuitive, selecting appropriate parameterisations for BRDF models is far from straightforward. We present a novel BRDF remapping technique, that automatically computes a mapping (BRDF Difference Probe) to match the appearance of a source material model to a target one. Through quantitative analysis, four user studies and psychometric scaling experiments, we validate our remapping framework and demonstrate that it yields a visually faithful remapping among analytical BRDFs. Most notably, our results show that even when the characteristics of the models are substantially different, such as in the case of a phenomenological model and a physically-based one, our remapped renderings are indistinguishable from the original source model.

Lightness Discrimination Depends More on Bright Rather Than Shaded Regions of Three-Dimensional Objects.

The brighter portions of a shaded complex object are in principle more informative about its lightness and are preferentially fixated during lightness judgments. In this study, we investigate whether preventing this strategy also has measurable detrimental effects on performance. Observers were presented with a reference and a comparison three-dimensional rendered object and had to choose which one was "painted with a lighter gray." The comparison was rendered with different diffuse reflectance values. We compared precision between three different conditions: full image, 20% of the lightest pixels removed, or 20% of the darkest pixels removed. Removing the bright pixels maximally impaired performance. The results confirm that the strategy of relying on the brightest areas of a complex object in order to estimate lightness is functionally optimal, yielding more precise representations.

Gloss and Speed Judgments Yield Different Fine Tuning of Saccadic Sampling in Dynamic Scenes.

Image motion contains potential cues about the material properties of objects. In earlier work, we proposed motion cues that could predict whether a moving object would be perceived as shiny or matte. However, whether the visual system uses these cues is still uncertain. Herein, we use the tracking of eye movements as a tool to understand what visual information observers use when engaged in material perception. Observers judged either the gloss or the speed of moving blobby shapes in an eye tracking experiment. Results indicate that during glossiness judgments, participants tend to look at gloss-diagnostic dynamic features more than during speed judgments. This suggests a fine tuning of the visual system to properties of moving stimuli: Task relevant information is actively singled out and processed in a dynamically changing environment.

Hyperspectral database of fruits and vegetables.

We have built a hyperspectral database of 42 fruits and vegetables. Both the outside (skin) and inside of the objects were imaged. We used a Specim VNIR HS-CL-30-V8E-OEM mirror-scanning hyperspectral camera and took pictures at a spatial resolution of ∼57 px/deg by 800 pixels at a wavelength resolution of ∼1.12 nm. A stable, broadband illuminant was used. Images and software are freely available on our webserver (http://www.allpsych.uni-giessen.de/GHIFVD; pronounced "gift"). We performed two kinds of analyses on these images. First, when comparing the insides and outsides of the objects, we observed that the insides were lighter than the skins, and that the hues of the insides and skins were significantly correlated (circular correlation=0.638). Second, we compared the color distribution within each object to corresponding human color discrimination thresholds. We found a significant correlation (0.75) between the orientation of ellipses fit to the chromaticity distributions of our fruits and vegetables with the orientations of interpolated MacAdam discrimination ellipses. This indicates a close relationship between sensory processing and the characteristics of environmental objects.

Categorizing natural color distributions.

The natural objects that we are surrounded with virtually always contain many different shades of color, yet the visual system usually categorizes them into a single color category. We examined various image statistics and their role in categorizing the color of leaves. Our subjects categorized photographs of autumn leaves and versions that were manipulated, including: randomly repositioned pixels, leaves uniformly colored with their mean color, leaves that were made by reflecting the original leaves' chromaticity distribution about their mean ("flipped leaves"), and simple patches colored with the mean colors of the original leaves. We trained a linear classifier with a set of image statistics in order to predict the category that each object was assigned to. Our results show that the mean hue of an object is highly predictive of the natural object's color category (>90% accuracy) and observers' choices are consistent with their use of unique yellow as a decision boundary for classification. The flipped leaves produced consistent changes in color categorization that are possibly explained by an interaction between the color distributions and the texture of the leaves.

Foveal to peripheral extrapolation of brightness within objects.

Peripheral viewing is characterized by poor resolution and distortions as compared to central viewing; nevertheless, when we move our gaze around, the visual scene does not appear to change. One possible mechanism leading to perceptual uniformity would be that peripheral appearance is extrapolated based on foveal information. Here we investigate foveal-to-peripheral extrapolation in the case of the perceived brightness of an object's surface. While fixating a spot on the rendered object, observers were asked to adjust the brightness of a disc to match a peripherally viewed target area on the surface of the same object. Being forced to fixate a better illuminated point led to brighter matches as compared to fixating points in the shadow, indicating that foveal brightness information was extrapolated. When observers fixated additional points outside of the object on the scene's background, fixated brightness had no effect on the brightness match. Results indicate that our visual system uses the brightness of the foveally viewed surface area to estimate the brightness of areas in the periphery. However, this mechanism is selectively applied within an object's boundary.

Seeing lightness in the dark.

From intense sunlight in bright snow down to a moonless night in a dark forest, we can use light to recognize objects and guide our actions. This remarkable range mainly rests on having two different types of photoreceptors, the rods and the cones. The cones are active under daylight conditions, allowing high acuity and color vision. Rods are mainly active under very dim illumination conditions and have an exquisite sensitivity to light [1]. There are obvious detriments to visual perception in near darkness, such as a central scotoma, reduced motion perception [2], and most of all a lack of color [3]. There is only one type of rod, and thus intensity and wavelength differences cannot be disentangled when only the rods are active. This is captured well by the old saying "at night all cats are gray", meaning that different colors inevitably get mapped onto different shades of gray. Here we show that the perception of lightness is also different for night vision: our results indicate that surfaces that appear to be white under daylight conditions, at best, appear medium gray under night vision, suggesting that activation of the cones is necessary for the perception of white.