Focusing on an illusion: Accommodating to perceived depth?

Ocular accommodation potentially provides information about depth but there is little evidence that this information is used by the human visual system. We use the hollow-face illusion, an illusion of depth reversal, to investigate whether accommodation is linked to perceived depth. In Experiment 1 accommodation, like vergence, was in front of the physical surface of the mask when the mask was upright and people reported experiencing the illusion. Accommodation to the illusory face did not differ significantly from accommodation to the physically convex back surface of the same mask. Only accommodation to the inverted mask seen as hollow was significantly less and, like the physical surface, beyond the mid-plane of the mask. The effect on accommodation was the same for monocular as binocular viewing, showing that accommodation is not driven by binocular disparities through vergence, although voluntary vergence remains a possibility. In Experiment 2 a projected random dot pattern was used to flip perception between convex and concave in all presentation conditions. Accommodation was again in front of the physical surface when the illusion was experienced. Experiment 3 showed that projected dots are more effective in disambiguating the illusion as concave when they are sharp and provide a good accommodative stimulus than when they are objectively blurred. We interpret Experiments 1 and 2 as showing that accommodation is tied to perceived depth, directly or indirectly, even in a situation where multiple depth cues are available and feedback is not artificially open-looped. Experiment 3 is consistent with accommodation helping to disambiguate depth while not ruling out alternative explanations.

About Face: Matching Unfamiliar Faces Across Rotations of View and Lighting.

Matching the identities of unfamiliar faces is heavily influenced by variations in their images. Changes to viewpoint and lighting direction during face perception are commonplace across yaw and pitch axes and can result in dramatic image differences. We report two experiments that, for the first time, factorially investigate the combined effects of lighting and view angle on matching performance for unfamiliar faces. The use of three-dimensional head models allowed control of both lighting and viewpoint. We found viewpoint effects in the yaw axis with little to no effect of lighting. However, for rotations about the pitch axis, there were both viewpoint and lighting effects and these interacted where lighting effects were found only for front views and views from below. The pattern of effects was similar regardless of whether view variation occurred as a result of head (Experiment 1) or camera (Experiment 2) suggesting that face matching is not purely image based. Along with face inversion effects in Experiment 1, the results of this study suggest that face perception is based on shape and surface information and draws on implicit knowledge of upright faces and ecological (top) lighting conditions.

The Nature and Timing of Tele-Pseudoscopic Experiences.

Interchanging the left and right eye views of a scene (pseudoscopic viewing) has been reported to produce vivid stereoscopic effects under certain conditions. In two separate field studies, we examined the experiences of 124 observers (76 in Study 1 and 48 in Study 2) while pseudoscopically viewing a distant natural outdoor scene. We found large individual differences in both the nature and the timing of their pseudoscopic experiences. While some observers failed to notice anything unusual about the pseudoscopic scene, most experienced multiple pseudoscopic phenomena, including apparent scene depth reversals, apparent object shape reversals, apparent size and flatness changes, apparent reversals of border ownership, and even complex illusory foreground surfaces. When multiple effects were experienced, patterns of co-occurrence suggested possible causal relationships between apparent scene depth reversals and several other pseudoscopic phenomena. The latency for experiencing pseudoscopic phenomena was found to correlate significantly with observer visual acuity, but not stereoacuity, in both studies.

Pinhole Viewing Strengthens the Hollow-Face Illusion.

A hollow (concave) mask appears convex when viewed from beyond a certain distance even when viewed stereoscopically-this is the hollow-face illusion. At close viewing distances, the same mask is seen as hollow even when disparity information is eliminated by monocular viewing. A potential source of nonpictorial, monocular information that favors a veridical percept at close distances is accommodation in conjunction with focus blur. In this article, we used pinhole viewing to minimize this potential source of information and test whether it affects whether a hollow mask is seen as veridical (concave) or illusory (convex). Since monocular viewing also facilitates the illusory (convex) percept, it was included in the design both as a comparison and to test whether any effect of accommodation depends on vergence. Pinhole viewing was found favor the illusory percept, and its effect was at least as large as, and added to, that of monocular viewing. A control experiment using tinted glasses that attenuate illumination at least as much as the pinholes did not strengthen the illusion ruling out explanations in terms of reduced luminance. For pinhole viewing, there was no difference between monocular and binocular conditions. The results are interpreted as evidence that focus driven depth information affects perceived three-dimensional shape at close distances even when other sources of depth information are available. The lack of a difference between monocular and binocular pinhole viewing suggests that, by disrupting accommodation, pinholes may also interfere with linked vergence cues to depth.

Toward DNA-based facial composites: preliminary results and validation.

The potential of constructing useful DNA-based facial composites is forensically of great interest. Given the significant identity information coded in the human face these predictions could help investigations out of an impasse. Although, there is substantial evidence that much of the total variation in facial features is genetically mediated, the discovery of which genes and gene variants underlie normal facial variation has been hampered primarily by the multipartite nature of facial variation. Traditionally, such physical complexity is simplified by simple scalar measurements defined a priori, such as nose or mouth width or alternatively using dimensionality reduction techniques such as principal component analysis where each principal coordinate is then treated as a scalar trait. However, as shown in previous and related work, a more impartial and systematic approach to modeling facial morphology is available and can facilitate both the gene discovery steps, as we recently showed, and DNA-based facial composite construction, as we show here. We first use genomic ancestry and sex to create a base-face, which is simply an average sex and ancestry matched face. Subsequently, the effects of 24 individual SNPs that have been shown to have significant effects on facial variation are overlaid on the base-face forming the predicted-face in a process akin to a photomontage or image blending. We next evaluate the accuracy of predicted faces using cross-validation. Physical accuracy of the facial predictions either locally in particular parts of the face or in terms of overall similarity is mainly determined by sex and genomic ancestry. The SNP-effects maintain the physical accuracy while significantly increasing the distinctiveness of the facial predictions, which would be expected to reduce false positives in perceptual identification tasks. To the best of our knowledge this is the first effort at generating facial composites from DNA and the results are preliminary but certainly promising, especially considering the limited amount of genetic information about the face contained in these 24 SNPs. This approach can incorporate additional SNPs as these are discovered and their effects documented. In this context we discuss three main avenues of research: expanding our knowledge of the genetic architecture of facial morphology, improving the predictive modeling of facial morphology by exploring and incorporating alternative prediction models, and increasing the value of the results through the weighted encoding of physical measurements in terms of human perception of faces.

Impaired perception of facial motion in autism spectrum disorder.

Facial motion is a special type of biological motion that transmits cues for socio-emotional communication and enables the discrimination of properties such as gender and identity. We used animated average faces to examine the ability of adults with autism spectrum disorders (ASD) to perceive facial motion. Participants completed increasingly difficult tasks involving the discrimination of (1) sequences of facial motion, (2) the identity of individuals based on their facial motion and (3) the gender of individuals. Stimuli were presented in both upright and upside-down orientations to test for the difference in inversion effects often found when comparing ASD with controls in face perception. The ASD group's performance was impaired relative to the control group in all three tasks and unlike the control group, the individuals with ASD failed to show an inversion effect. These results point to a deficit in facial biological motion processing in people with autism, which we suggest is linked to deficits in lower level motion processing we have previously reported.

Eye contrast polarity is critical for face recognition by infants.

Just as faces share the same basic arrangement of features, with two eyes above a nose above a mouth, human eyes all share the same basic contrast polarity relations, with a sclera lighter than an iris and a pupil, and this is unique among primates. The current study examined whether this bright-dark relationship of sclera to iris plays a critical role in face recognition from early in development. Specifically, we tested face discrimination in 7- and 8-month-old infants while independently manipulating the contrast polarity of the eye region and of the rest of the face. This gave four face contrast polarity conditions: fully positive condition, fully negative condition, positive face with negated eyes ("negative eyes") condition, and negated face with positive eyes ("positive eyes") condition. In a familiarization and novelty preference procedure, we found that 7- and 8-month-olds could discriminate between faces only when the contrast polarity of the eyes was preserved (positive) and that this did not depend on the contrast polarity of the rest of the face. This demonstrates the critical role of eye contrast polarity for face recognition in 7- and 8-month-olds and is consistent with previous findings for adults.

Recognising faces: effects of lighting direction, inversion, and brightness reversal.

When information about three-dimensional shape obtained from shading and shadows is ambiguous, the visual system favours an interpretation of surface geometry which is consistent with illumination from above. If pictures of top-lit faces are rotated the resulting stimulus is both figurally inverted and illuminated from below. In this study the question of whether the effects of figural inversion and lighting orientation on face recognition are independent or interactive is addressed. Although there was a clear inversion effect for faces illuminated from the front and above, the inversion effect was found to be reduced or eliminated for faces illuminated from below. A strong inversion effect for photographic negatives was also found but in this case the effect was not dependent on the direction of illumination. These findings are interpreted as evidence to suggest that lighting faces from below disrupts the formation of surface-based representations of facial shape.

Independent effects of local and global binocular disparity on the perceived convexity of stereoscopically presented faces in scenes.

Evidence suggests that experiencing the hollow-face illusion involves perceptual reversal of the binocular disparities associated with the face even though the rest of the scene appears unchanged. This suggests stereoscopic processing of object shape may be independent of scene-based processing of the layout of objects in depth. We investigated the effects of global scene-based and local object-based disparity on the compellingness of the perceived convexity of the face. We took stereoscopic photographs of people in scenes, and independently reversed the binocular disparities associated with the head and scene. Participants rated perceived convexity of a natural disparity ("convex") or reversed disparity ("concave") face shown either in its original context with reversed or natural disparities or against a black background. Faces with natural disparity were rated as more convincingly convex independent of the background, showing that the local disparities can affect perceived convexity independent of disparities across the rest of the image. However, the apparent convexity of the faces was also greater in natural disparity scenes compared to either a reversed disparity scene or a zero disparity black background. This independent effect of natural scene disparity suggests that the 'solidity' associated with natural scene disparities spread to enhance the perceived convexity of the face itself. Together, these findings suggest that global and local disparity exert independent and additive effects upon the perceived convexity of the face.

Perception of Mooney faces by young infants: the role of local feature visibility, contrast polarity, and motion.

We examined the ability of young infants (3- and 4-month-olds) to detect faces in the two-tone images often referred to as Mooney faces. In Experiment 1, this performance was examined in conditions of high and low visibility of local features and with either the presence or absence of the outer head contour. We found that regardless of the presence of the outer head contour, infants preferred upright over inverted two-tone face images only when local features were highly visible (Experiment 1a). We showed that this upright preference disappeared when the contrast polarity of two-tone images was reversed (Experiment 1b), reflecting operation of face-specific mechanisms. In Experiment 2, we investigated whether motion affects infants' perception of faces in Mooney faces. We found that when the faces appeared to be rigidly moving, infants did show an upright preference in conditions of low visibility of local features (Experiment 2a). Again the preference disappeared when the contrast polarity of the image was reversed (Experiment 2b). Together, these results suggest that young infants have the ability to integrate fragmented image features to perceive faces from two-tone face images, especially if they are moving. This suggests that an interaction between motion and form rather than a purely motion-based process (e.g., structure from motion) facilitates infants' perception of faces in ambiguous two-tone images.

Refractive error and monocular viewing strengthen the hollow-face illusion.

We measured the strength of the hollow-face illusion--the 'flipping distance' at which perception changes between convex and concave--as a function of a lens-induced 3 dioptre refractive error and monocular/binocular viewing. Refractive error and closing one eye both strengthened the illusion to approximately the same extent. The illusion was weakest viewed binocularly without refractive error and strongest viewed monocularly with it. This suggests binocular cues disambiguate the illusion at greater distances than monocular cues, but that both are disrupted by refractive error. We argue that refractive error leaves the ambiguous low-spatial-frequency shading information critical to the illusion largely unaffected while disrupting other, potentially disambiguating, depth/distance cues.

Binocular disparity magnitude affects perceived depth magnitude despite inversion of depth order.

The hollow-face illusion involves a misperception of depth order: our perception follows our top-down knowledge that faces are convex, even though bottom-up depth information reflects the actual concave surface structure. While pictorial cues can be ambiguous, stereopsis should unambiguously indicate the actual depth order. We used computer-generated stereo images to investigate how, if at all, the sign and magnitude of binocular disparities affect the perceived depth of the illusory convex face. In experiment 1 participants adjusted the disparity of a convex comparison face until it matched a reference face. The reference face was either convex or hollow and had binocular disparities consistent with an average face or had disparities exaggerated, consistent with a face stretched in depth. We observed that apparent depth increased with disparity magnitude, even when the hollow faces were seen as convex (ie when perceived depth order was inconsistent with disparity sign). As expected, concave faces appeared flatter than convex faces, suggesting that disparity sign also affects perceived depth. In experiment 2, participants were presented with pairs of real and illusory convex faces. In each case, their task was to judge which of the two stimuli appeared to have the greater depth. Hollow faces with exaggerated disparities were again perceived as deeper.

How Different is Different? Criterion and Sensitivity in Face-Space.

Not all detectable differences between face images correspond to a change in identity. Here we measure both sensitivity to change and the criterion difference that is perceived as a change in identity. Both measures are used to test between possible similarity metrics. Using a same/different task and the method of constant stimuli criterion is specified as the 50% "different" point (P50) and sensitivity as the difference limen (DL). Stimuli and differences are defined within a "face-space" based on principal components analysis of measured differences in three-dimensional shape. In Experiment 1 we varied views available. Criterion (P50) was lowest for identical full-face view comparisons that can be based on image differences. When comparing across views P50, was the same for a static 45° change as for multiple animated views, although sensitivity (DL) was higher for the animated case, where it was as high as for identical views. Experiments 2 and 3 tested possible similarity metrics. Experiment 2 contrasted Euclidean and Mahalanobis distance by setting PC1 or PC2 to zero. DL did not differ between conditions consistent with Mahalanobis. P50 was lower when PC2 changed emphasizing that perceived changes in identity are not determined by the magnitude of Euclidean physical differences. Experiment 3 contrasted a distance with an angle based similarity measure. We varied the distinctiveness of the faces being compared by varying distance from the origin, a manipulation that affects distances but not angles between faces. Angular P50 and DL were both constant for faces from 1 to 2 SD from the mean, consistent with an angular measure. We conclude that both criterion and sensitivity need to be considered and that an angular similarity metric based on standardized PC values provides the best metric for specifying what physical differences will be perceived to change in identity.

The hollow-face illusion in infancy: do infants see a screen based rotating hollow mask as hollow?

We investigated whether infants experience the hollow-face illusion using a screen-based presentation of a rotating hollow mask. In experiment 1 we examined preferential looking between rotating convex and concave faces. Adults looked more at the concave-illusory convex-face which appears to counter rotate. Infants of 7- to 8-month-old infants preferred the convex face, and 5- to 6-month-olds showed no preference. While older infants discriminate, their preference differed from that of adults possibly because they don't experience the illusion or counter rotation. In experiment 2 we tested preference in 7- to 8-month-olds for angled convex and concave static faces both before and after habituation to the stimuli shown in experiment 1. The infants showed a novelty preference for the static shape opposite to the habituation stimulus, together with a general preference for the static convex face. This shows that they discriminate between convex and concave faces and that habituation to either transfers across a change in view. Seven- to eight-month-olds have been shown to discriminate direction of rigid rotation on the basis of perspective changes. Our results suggest that this, perhaps together with a weaker bias to perceive faces as convex, allows these infants to see the screen-based hollow face as hollow even though adults perceive it as convex.

It's not what you say but the way you say it: matching faces and voices.

Recent studies have shown that the face and voice of an unfamiliar person can be matched for identity. Here the authors compare the relative effects of changing sentence content (what is said) and sentence manner (how it is said) on matching identity between faces and voices. A change between speaking a sentence as a statement and as a question disrupted matching performance, whereas changing the sentence itself did not. This was the case when the faces and voices were from the same race as participants and speaking a familiar language (English; Experiment 1) or from another race and speaking an unfamiliar language (Japanese; Experiment 2). Altering manner between conversational and clear speech (Experiment 3) or between conversational and casual speech (Experiment 4) was also disruptive. However, artificially slowing (Experiment 5) or speeding (Experiment 6) speech did not affect cross-modal matching performance. The results show that bimodal cues to identity are closely linked to manner but that content (what is said) and absolute tempo are not critical. Instead, prosodic variations in rhythmic structure and/or expressiveness may provide a bimodal, dynamic identity signature.

The hollow-face illusion: object-specific knowledge, general assumptions or properties of the stimulus?

The hollow-face illusion, in which a mask appears as a convex face, is a powerful example of binocular depth inversion occurring with a real object under a wide range of viewing conditions. Explanations of the illusion are reviewed and six experiments reported. In experiment 1 the detrimental effect of figural inversion, evidence for the importance of familiarity, was found for other oriented objects. The inversion effect held for masks lit from the side (experiment 2). The illusion was stronger for a mask rotated by 90 degrees lit from its forehead than from its chin, suggesting that familiar patterns of shading enhance the illusion (experiment 2). There were no effects of light source visibility or any left/right asymmetry (experiment 3). In experiments 4-6 we used a 'virtual' hollow face, with illusion strength quantified by the proportion of noise texture needed to eliminate the illusion. Adding characteristic surface colour enhanced the illusion, consistent with the familiar face pigmentation outweighing additional bottom-up cues (experiment 4). There was no difference between perspective and orthographic projection. Photographic negation reduced, but did not eliminate, the illusion, suggesting shading is important but not essential (experiment 5). Absolute depth was not critical, although a shallower mask was given less extreme convexity ratings (experiment 6). We argue that the illusion arises owing to a convexity preference when the raw data have ambiguous interpretations. However, using a familiar object with typical orientation, shading, and pigmentation greatly enhances the effect.

Infants' discrimination of faces by using biological motion cues.

We report two experiments in which we used animated averaged faces to examine infants' ability to perceive and discriminate facial motion. The faces were generated by using the motion recorded from the faces of volunteers while they spoke. We tested infants aged 4-8 months to assess their ability to discriminate facial motion sequences (condition 1) and discriminate the faces of individuals (condition 2). Infants were habituated to one sequence with the motion of one actor speaking one phrase. Following habituation, infants were presented with the same sequence together with motion from a different actor (condition 1), or a new sequence from the same actor coupled with a new sequence from a new actor (condition 2). Infants demonstrated a significant preference for the novel actor in both experiments. These findings suggest that infants can not only discriminate complex and subtle biological motion cues but also detect invariants in such displays.

Range- and domain-specific exaggeration of facial speech.

Is it possible to exaggerate the different ways in which people talk, just as we can caricature their faces? In this paper, we exaggerate animated facial movement to investigate how the emotional manner of speech is conveyed. Range-specific exaggerations selectively emphasized emotional manner whereas domain-specific exaggerations of differences in duration did not. Range-specific exaggeration relative to a time-locked average was more effective than absolute exaggeration of differences from the static, neutral face, despite smaller absolute differences in movement. Thus, exaggeration is most effective when the average used captures shared properties, allowing task-relevant differences to be selectively amplified. Playing the stimuli backwards showed that the effects of exaggeration were temporally reversible, although emotion-consistent ratings for stimuli played forwards were higher overall. Comparison with silent video showed that these stimuli also conveyed the intended emotional manner, that the relative rating of animations depends on the emotion, and that exaggerated animations were always rated at least as highly as video. Explanations in terms of key frame encoding and muscle-based models of facial movement are considered, as are possible methods for capturing timing-based cues.

"Putting the face to the voice": matching identity across modality.

Speech perception provides compelling examples of a strong link between auditory and visual modalities. This link originates in the mechanics of speech production, which, in shaping the vocal tract, determine the movement of the face as well as the sound of the voice. In this paper, we present evidence that equivalent information about identity is available cross-modally from both the face and voice. Using a delayed matching to sample task, XAB, we show that people can match the video of an unfamiliar face, X, to an unfamiliar voice, A or B, and vice versa, but only when stimuli are moving and are played forward. The critical role of time-varying information is underlined by the ability to match faces to voices containing only the coarse spatial and temporal information provided by sine wave speech [5]. The effect of varying sentence content across modalities was small, showing that identity-specific information is not closely tied to particular utterances. We conclude that the physical constraints linking faces to voices result in bimodally available dynamic information, not only about what is being said, but also about who is saying it.

Recognising facial expression from spatially and temporally modified movements.

We examined how the recognition of facial emotion was influenced by manipulation of both spatial and temporal properties of 3-D point-light displays of facial motion. We started with the measurement of 3-D position of multiple locations on the face during posed expressions of anger, happiness, sadness, and surprise, and then manipulated the spatial and temporal properties of the measurements to obtain new versions of the movements. In two experiments, we examined recognition of these original and modified facial expressions: in experiment 1, we manipulated the spatial properties of the facial movement, and in experiment 2 we manipulated the temporal properties. The results of experiment 1 showed that exaggeration of facial expressions relative to a fixed neutral expression resulted in enhanced ratings of the intensity of that emotion. The results of experiment 2 showed that changing the duration of an expression had a small effect on ratings of emotional intensity, with a trend for expressions with shorter durations to have lower ratings of intensity. The results are discussed within the context of theories of encoding as related to caricature and emotion.