Material category of visual objects computed from specular image structure.

Recognizing materials and their properties visually is vital for successful interactions with our environment, from avoiding slippery floors to handling fragile objects. Yet there is no simple mapping of retinal image intensities to physical properties. Here, we investigated what image information drives material perception by collecting human psychophysical judgements about complex glossy objects. Variations in specular image structure-produced either by manipulating reflectance properties or visual features directly-caused categorical shifts in material appearance, suggesting that specular reflections provide diagnostic information about a wide range of material classes. Perceived material category appeared to mediate cues for surface gloss, providing evidence against a purely feedforward view of neural processing. Our results suggest that the image structure that triggers our perception of surface gloss plays a direct role in visual categorization, and that the perception and neural processing of stimulus properties should be studied in the context of recognition, not in isolation.

Dynamic dot displays reveal material motion network in the human brain.

There is growing research interest in the neural mechanisms underlying the recognition of material categories and properties. This research field, however, is relatively more recent and limited compared to investigations of the neural mechanisms underlying object and scene category recognition. Motion is particularly important for the perception of non-rigid materials, but the neural basis of non-rigid material motion remains unexplored. Using fMRI, we investigated which brain regions respond preferentially to material motion versus other types of motion. We introduce a new database of stimuli - dynamic dot materials - that are animations of moving dots that induce vivid percepts of various materials in motion, e.g. flapping cloth, liquid waves, wobbling jelly. Control stimuli were scrambled versions of these same animations and rigid three-dimensional rotating dots. Results showed that isolating material motion properties with dynamic dots (in contrast with other kinds of motion) activates a network of cortical regions in both ventral and dorsal visual pathways, including areas normally associated with the processing of surface properties and shape, and extending to somatosensory and premotor cortices. We suggest that such a widespread preference for material motion is due to strong associations between stimulus properties. For example viewing dots moving in a specific pattern not only elicits percepts of material motion; one perceives a flexible, non-rigid shape, identifies the object as a cloth flapping in the wind, infers the object's weight under gravity, and anticipates how it would feel to reach out and touch the material. These results are a first important step in mapping out the cortical architecture and dynamics in material-related motion processing.

Expectations affect the perception of material properties.

Many objects that we encounter have typical material qualities: spoons are hard, pillows are soft, and Jell-O dessert is wobbly. Over a lifetime of experiences, strong associations between an object and its typical material properties may be formed, and these associations not only include how glossy, rough, or pink an object is, but also how it behaves under force: we expect knocked over vases to shatter, popped bike tires to deflate, and gooey grilled cheese to hang between two slices of bread when pulled apart. Here we ask how such rich visual priors affect the visual perception of material qualities and present a particularly striking example of expectation violation. In a cue conflict design, we pair computer-rendered familiar objects with surprising material behaviors (a linen curtain shattering, a porcelain teacup wrinkling, etc.) and find that material qualities are not solely estimated from the object's kinematics (i.e., its physical [atypical] motion while shattering, wrinkling, wobbling etc.); rather, material appearance is sometimes "pulled" toward the "native" motion, shape, and optical properties that are associated with this object. Our results, in addition to patterns we find in response time data, suggest that visual priors about materials can set up high-level expectations about complex future states of an object and show how these priors modulate material appearance.

Shatter and splatter: The contribution of mechanical and optical properties to the perception of soft and hard breaking materials.

Research on the visual perception of materials has mostly focused on the surface qualities of rigid objects. The perception of substance like materials is less explored. Here, we investigated the contribution of, and interaction between, surface optics and mechanical properties to the perception of nonrigid, breaking materials. We created novel animations of materials ranging from soft to hard bodies that broke apart differently when dropped. In Experiment 1, animations were rendered as point-light movies varying in dot density, as well as "full-cue" optical versions ranging from translucent glossy to opaque matte under a natural illumination field. Observers used a scale to rate each substance on different attributes. In Experiment 2 we investigated how much shape contributed to ratings of the full-cue stimuli in Experiment 1, by comparing ratings when observers were shown movies versus one frame of the animation. The results showed that optical and mechanical properties had an interactive effect on ratings of several material attributes. We also found that motion and static cues each provided a lot of information about the material qualities; however, when combined, they influenced observers' ratings interactively. For example, in some conditions, motion dominated over optical information; in other conditions, it enhanced the effect of optics. Our results suggest that rating multiple attributes is an effective way to measure underlying perceptual differences between nonrigid breaking materials, and this study is the first to our knowledge to show interactions between optical and mechanical properties in a task involving judgments of perceptual qualities.

Perceptual dimensions underlying lightness perception in homogeneous center-surround displays.

Lightness judgments of targets embedded in a homogeneous surround exhibit abrupt steps in perceived lightness at points at which the targets transition from being increments to decrements. This "crispening effect" and the general difficulty of matching low-contrast targets embedded in homogeneous surrounds suggest that a second perceptual dimension in addition to lightness may contribute to the appearance of test patches in these displays. The present study explicitly tested whether two dimensions (lightness and transmittance) could lead to more satisfactory matches than lightness alone in an asymmetric matching task. We also examined whether transmittance matches were more strongly associated with task instructions that had observers match perceived transparency or the perceived edge contrast of the target relative to the surround. We found that matching target lightness in a homogeneous display to that in a textured or rocky display required varying both lightness and transmittance of the test patch on the textured display to obtain the most satisfactory matches. However, observers primarily varied transmittance when instructed to match the perceived contrast of targets against homogeneous surrounds, but not when instructed to match the amount of transparency perceived in the displays. The results suggest that perceived target-surround edge contrast differs between homogeneous and textured displays. Varying the midlevel property of transparency in textured displays provides a natural means for equating both target lightness and the unique appearance of the edge contrast in homogeneous displays.

Error Rates in Users of Automatic Face Recognition Software.

In recent years, wide deployment of automatic face recognition systems has been accompanied by substantial gains in algorithm performance. However, benchmarking tests designed to evaluate these systems do not account for the errors of human operators, who are often an integral part of face recognition solutions in forensic and security settings. This causes a mismatch between evaluation tests and operational accuracy. We address this by measuring user performance in a face recognition system used to screen passport applications for identity fraud. Experiment 1 measured target detection accuracy in algorithm-generated 'candidate lists' selected from a large database of passport images. Accuracy was notably poorer than in previous studies of unfamiliar face matching: participants made over 50% errors for adult target faces, and over 60% when matching images of children. Experiment 2 then compared performance of student participants to trained passport officers-who use the system in their daily work-and found equivalent performance in these groups. Encouragingly, a group of highly trained and experienced "facial examiners" outperformed these groups by 20 percentage points. We conclude that human performance curtails accuracy of face recognition systems-potentially reducing benchmark estimates by 50% in operational settings. Mere practise does not attenuate these limits, but superior performance of trained examiners suggests that recruitment and selection of human operators, in combination with effective training and mentorship, can improve the operational accuracy of face recognition systems.

Do surface reflectance properties and 3-D mesostructure influence the perception of lightness?

A series of experiments were conducted to assess how the reflectance properties and the complexity of surface "mesostructure" (small-scale 3-D relief) influence perceived lightness. Experiment 1 evaluated the role of surface relief and gloss on perceived lightness. For surfaces with visible mesostructure, lightness constancy was better for targets embedded in glossy than matte surfaces. The results for surfaces that lacked surface relief were qualitatively different than the 3-D surrounds, exhibiting abrupt steps in perceived lightness at points at which the targets transition from being increments to decrements. Experiments 2 and 4 compared the matte and glossy 3-D surrounds to two control displays, which matched either pixel histograms or a phase-scrambled power spectrum, respectively. Although some improved lightness constancy was observed for the 3-D gloss display over the histogram-matched display, this benefit was not observed for phase-scrambled variants of these images with equated power spectrums. These results suggest that the improved lightness constancy observed with 3-D surfaces can be well explained by the distribution of contrast across space and scale, independently of explicit information about surface shading or specularity whereas the putatively "simpler" flat displays may evoke more complex midlevel representations similar to that evoked in conditions of transparency.

The role of amodal surface completion in stereoscopic transparency.

Previous work has shown that the visual system can decompose stereoscopic textures into percepts of inhomogeneous transparency. We investigate whether this form of layered image decomposition is shaped by constraints on amodal surface completion. We report a series of experiments that demonstrate that stereoscopic depth differences are easier to discriminate when the stereo images generate a coherent percept of surface color, than when images require amodally integrating a series of color changes into a coherent surface. Our results provide further evidence for the intimate link between the segmentation processes that occur in conditions of transparency and occlusion, and the interpolation processes involved in the formation of amodally completed surfaces.