Are people with autism prosopagnosic?

Difficulties in various face processing tasks have been well documented in autism spectrum disorder (ASD). Several meta-analyses and numerous case-control studies have indicated that this population experiences a moderate degree of impairment, with a small percentage of studies failing to detect any impairment. One possible account of this mixed pattern of findings is heterogeneity in face processing abilities stemming from the presence of a subpopulation of prosopagnosic individuals with ASD alongside those with normal face processing skills. Samples randomly drawn from such a population, especially relatively smaller ones, would vary in the proportion of participants with prosopagnosia, resulting in a wide range of group-level deficits from mild (or none) to severe across studies. We test this prosopagnosic subpopulation hypothesis by examining three groups of participants: adults with ASD, adults with developmental prosopagnosia (DP), and a comparison group. Our results show that the prosopagnosic subpopulation hypothesis does not account for the face impairments in the broader autism spectrum. ASD observers show a continuous and graded, rather than categorical, heterogeneity that span a range of face processing skills including many with mild to moderate deficits, inconsistent with a prosopagnosic subtype account. We suggest that pathogenic origins of face deficits for at least some with ASD differ from those of DP.

Artificial intelligence, explainability, and the scientific method: A proof-of-concept study on novel retinal biomarker discovery.

We present a structured approach to combine explainability of artificial intelligence (AI) with the scientific method for scientific discovery. We demonstrate the utility of this approach in a proof-of-concept study where we uncover biomarkers from a convolutional neural network (CNN) model trained to classify patient sex in retinal images. This is a trait that is not currently recognized by diagnosticians in retinal images, yet, one successfully classified by CNNs. Our methodology consists of four phases: In Phase 1, CNN development, we train a visual geometry group (VGG) model to recognize patient sex in retinal images. In Phase 2, Inspiration, we review visualizations obtained from post hoc interpretability tools to make observations, and articulate exploratory hypotheses. Here, we listed 14 hypotheses retinal sex differences. In Phase 3, Exploration, we test all exploratory hypotheses on an independent dataset. Out of 14 exploratory hypotheses, nine revealed significant differences. In Phase 4, Verification, we re-tested the nine flagged hypotheses on a new dataset. Five were verified, revealing (i) significantly greater length, (ii) more nodes, and (iii) more branches of retinal vasculature, (iv) greater retinal area covered by the vessels in the superior temporal quadrant, and (v) darker peripapillary region in male eyes. Finally, we trained a group of ophthalmologists (N=26) to recognize the novel retinal features for sex classification. While their pretraining performance was not different from chance level or the performance of a nonexpert group (N=31), after training, their performance increased significantly (p<0.001, d=2.63). These findings showcase the potential for retinal biomarker discovery through CNN applications, with the added utility of empowering medical practitioners with new diagnostic capabilities to enhance their clinical toolkit.

Learning from small data: Classifying sex from retinal images via deep learning.

Deep learning (DL) techniques have seen tremendous interest in medical imaging, particularly in the use of convolutional neural networks (CNNs) for the development of automated diagnostic tools. The facility of its non-invasive acquisition makes retinal fundus imaging particularly amenable to such automated approaches. Recent work in the analysis of fundus images using CNNs relies on access to massive datasets for training and validation, composed of hundreds of thousands of images. However, data residency and data privacy restrictions stymie the applicability of this approach in medical settings where patient confidentiality is a mandate. Here, we showcase results for the performance of DL on small datasets to classify patient sex from fundus images-a trait thought not to be present or quantifiable in fundus images until recently. Specifically, we fine-tune a Resnet-152 model whose last layer has been modified to a fully-connected layer for binary classification. We carried out several experiments to assess performance in the small dataset context using one private (DOVS) and one public (ODIR) data source. Our models, developed using approximately 2500 fundus images, achieved test AUC scores of up to 0.72 (95% CI: [0.67, 0.77]). This corresponds to a mere 25% decrease in performance despite a nearly 1000-fold decrease in the dataset size compared to prior results in the literature. Our results show that binary classification, even with a hard task such as sex categorization from retinal fundus images, is possible with very small datasets. Our domain adaptation results show that models trained with one distribution of images may generalize well to an independent external source, as in the case of models trained on DOVS and tested on ODIR. Our results also show that eliminating poor quality images may hamper training of the CNN due to reducing the already small dataset size even further. Nevertheless, using high quality images may be an important factor as evidenced by superior generalizability of results in the domain adaptation experiments. Finally, our work shows that ensembling is an important tool in maximizing performance of deep CNNs in the context of small development datasets.

Face perception: A brief journey through recent discoveries and current directions.

Faces are a rich source of information about the people around us. Identity, state of mind, emotions, intentions, age, gender, ethnic background, attractiveness and a host of other attributes about an individual can be gleaned from a face. When face perception fails, dramatic psycho-social consequences can follow at the individual level, as in the case of prosopagnosic parents who are unable to recognize their children at school pick-up. At the species level, social interaction patterns are shaped by human face perception abilities. The computational feat of recognizing faces and facial attributes, and the challenges overcome by the human brain to achieve this feat, have fascinated generations of vision researchers. In this paper, we present a brief overview of some of the milestones of discovery as well as outline a selected set of current directions and open questions on this topic.

The adult face-diet: A naturalistic observation study.

Experience plays a fundamental role in the development of visual function. Exposure to different types of faces is an important factor believed to shape face perception ability. Contents of daily exposure to faces, i.e., the face-diet, of infants have been documented in previous studies. While face perception involves a protracted development and continues to be malleable well into adulthood, an empirical study of the adult face-diet has been lacking. We collected first-person perspective footage from 30 adults during the course of their daily activities. We found that adults' exposure to faces is longer and more diverse compared to that of infants. Frequency of exposure were highest for familiar (75%), own-race (81%), and three-quarter pose (44%) faces. Faces in the adult face-diet were relatively large (median 6°) suggesting fairly close viewing distances. Face sizes were significantly larger for familiar (median 7.1°) compared to unfamiliar (median 4.9°) faces, reflecting the closer viewing distances that characterize social interaction. These results are consistent with the view that face recognition processes are tuned to the ecologically relevant values of face attributes that are encountered most frequently in the real-life context to optimize face perception abilities.

Perceptual efficiency and the inversion effect for faces, words and houses.

Face and visual word recognition are two key forms of expert visual processing. In the domain of object recognition, it has been suggested that expert processing is characterized by the use of different mechanisms from the ones involved in general object recognition. It has been suggested that one traditional marker of expert processing is the inversion effect. To investigate whether face and word recognition differ from general object recognition, we compared the effect of inversion on the perceptual efficiency of face and visual word recognition as well as on the recognition of a third, non-expert object category, houses. From the comparison of identification contrast thresholds to an ideal observer, we derived the efficiency and equivalent input noise of stimulus processing in both upright and inverted orientations. While efficiency reflects the efficacy in sampling the available information, equivalent input noise is associated with the degradation of the stimulus signal within the visual system. We hypothesized that large inversion effects for efficiency and/or equivalent input noise should characterize expert high-level processes, and asked whether this would be true for both faces and words, but not houses. However, we found that while face recognition efficiency was profoundly reduced by inversion, the efficiency of word and house recognition was minimally influenced by the orientation manipulation. Inversion did not affect equivalent input noise. These results suggest that even though faces and words are both considered expert processes, only the efficiency of the mechanism involved in face recognition is sensitive to orientation.

Link Between Facial Identity and Expression Abilities Suggestive of Origins of Face Impairments in Autism: Support for the Social-Motivation Hypothesis.

Individuals with autism spectrum disorder (ASD) often have difficulties with processing identity and expression in faces. This is at odds with influential models of face processing that propose separate neural pathways for the identity and expression domains. The social-motivation hypothesis of ASD posits a lack of visual experience with faces as the root cause of face impairments in autism. A direct prediction is that identity and expression abilities should be related in ASD, reflecting the common origin of face impairment in this population. We tested adults with and without ASD ( ns = 34) in identity and expression tasks. Our results showed that performance in the two domains was significantly correlated in the ASD group but not in the comparison group. These results suggest that the most likely origin for face impairments in ASD stems from the input stage impacting development of identity and expression domains alike, consistent with the social-motivation hypothesis.

Qualitatively similar processing for own- and other-race faces: Evidence from efficiency and equivalent input noise.

The other-race effect is the finding of diminished performance in recognition of other-race faces compared to those of own-race. It has been suggested that the other-race effect stems from specialized expert processes being tuned exclusively to own-race faces. In the present study, we measured recognition contrast thresholds for own- and other-race faces as well as houses for Caucasian observers. We have factored face recognition performance into two invariant aspects of visual function: efficiency, which is related to neural computations and processing demanded by the task, and equivalent input noise, related to signal degradation within the visual system. We hypothesized that if expert processes are available only to own-race faces, this should translate into substantially greater recognition efficiencies for own-race compared to other-race faces. Instead, we found similar recognition efficiencies for both own- and other-race faces. The other-race effect manifested as increased equivalent input noise. These results argue against qualitatively distinct perceptual processes. Instead they suggest that for Caucasian observers, similar neural computations underlie recognition of own- and other-race faces.

Visual orientation processing in autism spectrum disorder: No sign of enhanced early cortical function.

It has been suggested that enhanced perceptual processing underlies some of the social difficulties associated with autism spectrum disorder (ASD). While a variety of visual tasks have been reported in which individuals with ASD outperform neurotypical individuals in control groups, the precise origin of such effects within the visual pathway remains unclear. It has recently been established that visual acuity is intact yet unremarkable in ASD. This suggests that the earliest levels of retinal processing are an unlikely candidate as the source of differences. The next potential levels for divergent visual processing are those involved in processing simple aspects of visual stimuli, such as orientation and spatial frequency, considered to be functions of early visual cortex. Here we focused on the basic processing of orientation. In three experiments, we assessed three basic aspects of orientation processing-discrimination, veridical perception, and detection-in participants with ASD in comparison to age-, gender-, and IQ-matched adults without ASD. Each experiment allowed for both qualitative and quantitative comparisons between the two groups. These provided a dense array of data indicating that participants with ASD perceive orientation of low-level stimuli in a qualitatively (as well as quantitatively) similar manner to participants without ASD in control groups, with no evidence of superior processing in detection, precision, or accuracy aspects of orientation perception. These results suggest that the source for altered perceptual abilities should be sought elsewhere, possibly in specific subgroups of people with ASD, other aspects of low-level vision such as spatial frequency, or subsequent levels of visual processing.

Interactions between the perception of age and ethnicity in faces: an event-related potential study.

Face perception models propose that different facial attributes are processed by anatomically distinct neural pathways that partially overlap. Whether these attributes interact functionally is an open question. Our goal was to determine if there are interactions between age and ethnicity processing and, if so, at what temporal epoch these interactions are evident. We monitored event-related potentials on electroencephalography while subjects categorized faces by age or ethnicity in two conditions: a baseline in which the other of these two properties not being categorized was held constant and an interference condition in which it also varied, as modelled after the Garner interference paradigm. We found that, when participants were categorizing faces by age, variations in ethnicity increased the amplitude of the right face-selective N170 component. When subjects were categorizing faces by ethnicity, variations in age did not alter the N170. We concluded that there is an asymmetric pattern of influence between age and ethnicity on early face-specific stages of visual processing, which has parallels with behavioural evidence of asymmetric interactions between identity and expression processing of faces.

Temporal dynamics of the face familiarity effect: bootstrap analysis of single-subject event-related potential data.

Prior event-related potential studies using group statistics within a priori selected time windows have yielded conflicting results about familiarity effects in face processing. Our goal was to evaluate the temporal dynamics of the familiarity effect at all time points at the single-subject level. Ten subjects were shown faces of anonymous people or celebrities. Individual results were analysed using a point-by-point bootstrap analysis. While familiarity effects were less consistent at later epochs, all subjects showed them between 130 and 195 ms in occipitotemporal electrodes. However, the relation between the time course of familiarity effects and the peak latency of the N170 was variable. We concluded that familiarity effects between 130 and 195 ms are robust and can be shown in single subjects. The variability of their relation to the timing of the N170 potential may lead to underestimation of familiarity effects in studies that use group-based statistics.

Size determines whether specialized expert processes are engaged for recognition of faces.

Many influential models of face recognition postulate specialized expert processes that are engaged when viewing upright, own-race faces, as opposed to a general-purpose recognition route used for nonface objects and inverted or other-race faces. In contrast, others have argued that empirical differences do not stem from qualitatively distinct processing. We offer a potential resolution to this ongoing controversy. We hypothesize that faces engage specialized processes at large sizes only. To test this, we measured recognition efficiencies for a wide range of sizes. Upright face recognition efficiency increased with size. This was not due to better visibility of basic image features at large sizes. We ensured this by calculating efficiency relative to a specialized ideal observer unique to each individual that incorporated size-related changes in visibility and by measuring inverted efficiencies across the same range of face sizes. Inverted face recognition efficiencies did not change with size. A qualitative face inversion effect, defined as the ratio of relative upright and inverted efficiencies, showed a complete lack of inversion effects for small sizes up to 6°. In contrast, significant face inversion effects were found for all larger sizes. Size effects may stem from predominance of larger faces in the overall exposure to faces, which occur at closer viewing distances typical of social interaction. Our results offer a potential explanation for the contradictory findings in the literature regarding the special status of faces.

Spontaneous perceptual facial distortions correlate with ventral occipitotemporal activity.

Prosopometamorphopsia is a disorder of face perception in which faces appear distorted to the perceiver. The neural basis of prosopometamorphopsia is unclear, but may involve abnormal activity in face-selective areas in the ventral occipito-temporal pathway. Here we present the case of AS, a 44-year-old woman who reports persistent perceptual distortions of faces with no known cause. AS was presented with facial images and rated the magnitude of her distortions while activity in her core face areas and other areas in the ventral visual pathway was measured using functional magnetic resonance imaging. The magnitude of her distortions was positively correlated with signal changes in the right occipital face area (OFA) and right fusiform face area (FFA), as well as right V1-V3, and right lateral occipital cortex (LOC). There was also a trend for a significant correlation with signal in the left OFA and right inferior frontal gyrus (IFG), but not in the right or left superior temporal sulcus (STS). These results suggest that AS' prosopometamorphopsia reflects anomalous activity in face-processing network, particularly in the ventral occipitotemporal cortex.

Looking at a blurry old family photo? Zoom out!

We investigated recognition of blurry faces and whether viewing size affects identification of such severely degraded images. Despite the common belief that face perception relies on middle spatial frequencies, the critical spatial frequency band for face recognition is not fixed but rather depends on size. This is especially pronounced at small sizes, where observers choose to utilize lower, rather than middle, frequencies to identify a face. Here we assessed recognition of identity via a novel use of the face adaptation paradigm. We examined face identity aftereffects of blurry and intact adaptors at two sizes. Intact adaptors induced significant aftereffects regardless of size. Small, but not large, blurry adaptors produced aftereffects despite the fact that both contained exactly the same level of facial detail. This suggests an inability to utilize low-frequency information for perceiving identity in large faces. We conclude that (1) size is a key factor in human face recognition processes and (2) coarse facial images are better recognized at small sizes.

Neuroanatomic correlates of the feature-salience hierarchy in face processing: an fMRI -adaptation study.

Previous fMRI studies suggest that faces are represented holistically in human face processing regions. On the other hand, behavioral studies have also shown that some facial features are more salient than others for face recognition: the neural basis of this feature-salience hierarchy is not known. We used fMRI-adaptation together with a behavioral discrimination task and an ideal observer analysis to ask (1) whether different face parts contribute different amounts to the neural signal in face responsive regions, and (2) whether this response correlates more with the behavioral performance of human subjects or with the physical properties of the face stimuli. Twenty-three subjects performed a same/different discrimination experiment to characterize their ability to detect changes to different face parts. The same subjects underwent an fMRI-adaptation study, in which limited portions of the faces were repeated or changed between alternating stimuli. The behavioral study showed high efficiency in identity discrimination when the whole face, top half, or eyes changed, and low efficiency when the bottom half, nose, or mouth changed. During fMRI, there was a release of adaptation in the right and left fusiform face area (FFA) with changes to the whole face, top face-half, or the eyes. Changes to the bottom half, nose or mouth did not result in a significant release of adaptation in the right FFA, although bottom-half changes resulted in a release of adaptation in the left FFA. Adaptation in the right and left FFA and the right pSTS was correlated with human perceptual efficiency but not with ideal observer measures of the physical image differences between face parts. The feature-salience hierarchy of human face perception is therefore reflected in the activity in the right and left FFA and right pSTS, further supporting the key role of these structures in our perceptual experience of faces.

The role of skin texture and facial shape in representations of age and identity.

Faces have both shape and skin texture, but the relative importance of the two in face representations is unclear. Our goals were first, to determine the contribution of shape versus texture to aftereffects for facial age and identity and second, to assess whether adaptation transferred between shape and texture, suggesting integration in a single representation. In our first experiment we examined age aftereffects. We obtained young and old images of two celebrities and created hybrid images, one combining the structure of the old face with the skin texture of the young face, the other combining the young structure with the old skin texture. This allowed us to create adaptation contrasts where the two adapting faces had the same facial structure but different skin texture, and vice versa. In the second experiment, we performed a similar study but this time examining identity aftereffects between two people of a similar age. We found that both skin texture and facial shape generated significant age aftereffects, but the contribution was greater from texture than from shape. Both texture and shape also generated significant identity aftereffects, but the contribution was greater from shape than from texture. In the last experiment, we used the normal and hybrid images to determine if adaptation to one property (i.e., texture) could create aftereffects in the perception of age in the other property (i.e., shape). While there was significant within-component adaptation for texture and shape, there was no evidence of cross-component adaptation. We conclude that shape and texture contribute differently to different face representations, with texture dominating for age. The lack of cross-component adaptation transfer suggests independent encoding of shape and texture, at least for age representations.

Facial age after-effects show partial identity invariance and transfer from hands to faces.

Age imparts long-term dynamic changes to faces: how these are represented in the human visual system has seldom been investigated. We investigated facial age after-effects using a perceptual bias paradigm, and studied the ability of adaptation to transfer across face identity, visual stimuli and sensory modality, as has been done for the short-term dynamic changes of facial expression. Age after-effects were reduced but still significant when the identity of the face was changed between the adapting and test stimuli, as we had found for expression after-effects, suggesting identity-specific and identity-invariant components of age after-effects. Although body silhouettes and greyscale body images failed to generate age after-effects in faces, we did find cross-stimulus transfer of age adaptation from hands to faces. There was no cross-modal transfer of after-effects from voices to faces. These findings confirm that face adaptation has components that cannot be explained by low-level image-based effects but involve high-level representations that may be influenced by related visual semantic information.

The nature of upright and inverted face representations: an adaptation-transfer study of configuration.

It is considered that whole-face processing of spatial structure may only be possible in upright faces, with only local feature processing in inverted faces. We asked whether this was due to impoverished representations of inverted faces. We performed two experiments. In the first, we divided faces into segments to create 'exploded' faces with disrupted second-order structures, and 'scrambled' faces with altered first-order relations; in the second we shifted features within intact facial outlines to create equivalent disruptions of spatial structure. In both we assessed the transfer of adaptation between faces with altered structure and intact faces. Scrambled adaptors did not adapt upright or inverted intact faces, indicating that a whole-face configuration is required at either orientation. Both upright and inverted faces showed a similar decline in aftereffect magnitude when adapting faces had altered second-order structure, implying that this structure is present in both upright and inverted face representations. We conclude that inverted faces are not represented simply as a collection of features, but have a whole-face configuration with second-order structure, similar to upright faces. Thus the qualitative impairments induced by inversion are not due to degraded inverted facial representations, but may reflect limitations in perceptual mechanisms.