Neural Representations of Observed Interpersonal Synchrony/Asynchrony in the Social Perception Network.

The visual perception of individuals is thought to be mediated by a network of regions in the occipitotemporal cortex that supports specialized processing of faces, bodies, and actions. In comparison, we know relatively little about the neural mechanisms that support the perception of multiple individuals and the interactions between them. The present study sought to elucidate the visual processing of social interactions by identifying which regions of the social perception network represent interpersonal synchrony. In an fMRI study with 32 human participants (26 female, 6 male), we used multivoxel pattern analysis to investigate whether activity in face-selective, body-selective, and interaction-sensitive regions across the social perception network supports the decoding of synchronous versus asynchronous head-nodding and head-shaking. Several regions were found to support significant decoding of synchrony/asynchrony, including extrastriate body area (EBA), face-selective and interaction-sensitive mid/posterior right superior temporal sulcus, and occipital face area. We also saw robust cross-classification across actions in the EBA, suggestive of movement-invariant representations of synchrony/asynchrony. Exploratory whole-brain analyses also identified a region of the right fusiform cortex that responded more strongly to synchronous than to asynchronous motion. Critically, perceiving interpersonal synchrony/asynchrony requires the simultaneous extraction and integration of dynamic information from more than one person. Hence, the representation of synchrony/asynchrony cannot be attributed to augmented or additive processing of individual actors. Our findings therefore provide important new evidence that social interactions recruit dedicated visual processing within the social perception network that extends beyond that engaged by the faces and bodies of the constituent individuals.

Recognition of facial expressions in autism: Effects of face masks and alexithymia.

It is often assumed that the recognition of facial expressions is impaired in autism. However, recent evidence suggests that reports of expression recognition difficulties in autistic participants may be attributable to co-occurring alexithymia-a trait associated with difficulties interpreting interoceptive and emotional states-not autism per se. Due to problems fixating on the eye-region, autistic individuals may be more reliant on information from the mouth region when judging facial expressions. As such, it may be easier to detect expression recognition deficits attributable to autism, not alexithymia, when participants are forced to base expression judgements on the eye-region alone. To test this possibility, we compared the ability of autistic participants (with and without high levels of alexithymia) and non-autistic controls to categorise facial expressions (a) when the whole face was visible, and (b) when the lower portion of the face was covered with a surgical mask. High-alexithymic autistic participants showed clear evidence of expression recognition difficulties: they correctly categorised fewer expressions than non-autistic controls. In contrast, low-alexithymic autistic participants were unimpaired relative to non-autistic controls. The same pattern of results was seen when judging masked and unmasked expression stimuli. In sum, we find no evidence for an expression recognition deficit attributable to autism, in the absence of high levels of co-occurring alexithymia, either when participants judge whole-face stimuli or just the eye-region. These findings underscore the influence of co-occurring alexithymia on expression recognition in autism.

Does a lack of perceptual expertise prevent participants from forming reliable first impressions of "other-race" faces?

[Correction Notice: An Erratum for this article was reported online in Journal of Experimental Psychology: General on Feb 27 2023 (see record 2023-48793-001). The original article is being made available open access under the CC-BY license following the University of Nottingham opt-in to the Jisc/APA Read and Publish agreement. The correct copyright is "© 2022 The Author(s)" and the CC-BY license disclaimer is below. All versions of this article have been corrected. "Open Access funding provided by Birkbeck, University of London: This work is licensed under a Creative Commons Attribution 4.0 International License (CC-BY). This license permits copying and redistributing the work in any medium or format, as well as adapting the material for any purpose, even commercially."] Many studies investigating first impressions from faces employ stimulus sets that comprise only White faces. It is argued that participants lack the necessary perceptual expertise to provide reliable trait evaluations when viewing faces from ethnicities that differ from their own. In combination with a reliance on White and WEIRD participants, this concern has contributed to the widespread use of White face stimuli in this literature. The present study sought to determine whether concerns about the use of so-called other-race faces are justified by assessing the test-retest reliability of trait judgments made about same- and other-race faces. In two experiments conducted on 400 British participants, we find that White British participants made reliable trait judgments about Black faces, and Black British participants made reliable trait judgments about White faces. It is important that future work be conducted to determine how widely these results generalize. Considering our findings, however, we suggest (a) that the default assumption in future first impressions research should be that participants-particularly those recruited from diverse communities-are able to form reliable first impressions of other-race faces and (b) that faces of color be included in stimulus sets whenever possible. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Preferential looking studies of trustworthiness detection confound structural and expressive cues to facial trustworthiness.

On encountering a stranger, we spontaneously attribute to them character traits (e.g., trustworthiness, intelligence) based on their facial appearance. Participants can base impressions on structural face cues-the stable aspects of facial appearance that support identity recognition-or expression cues, such as the presence of a smile. It has been reported that 6- to 8-month-old infants attend to faces that adults judge to be trustworthy in preference to faces judged untrustworthy. These results are striking because the face stimuli employed were ostensibly emotion neutral. Consequently, these preferential looking effects have been taken as evidence for innate sensitivity to structural face cues to trustworthiness. However, scrutiny of the emotion rating procedure used with adults suggests that the face stimuli employed may have been judged emotion neutral only when interleaved with more obvious examples of facial affect. This means that the faces may vary in emotional expression when compared to each other. Here, we report new evidence obtained from adult raters that the stimuli used in these studies confound trustworthiness and untrustworthiness with the presence of happiness and anger, respectively. These findings suggest that the preferential looking effects described in infants are compatible with a preference for positive facial affect and may not reflect early sensitivity to structural face cues to trustworthiness.

New evidence of impaired expression recognition in developmental prosopagnosia.

Developmental prosopagnosia (DP) is a neurodevelopmental condition characterized by lifelong face recognition difficulties. To date, it remains unclear whether or not individuals with DP experience impaired recognition of facial expressions. It has been proposed that DPs may have sufficient perceptual ability to correctly interpret facial expressions when tasks are relatively easy (e.g., the stimuli are unambiguous and viewing conditions are optimal), but exhibit subtle impairments when tested under more challenging conditions. In the present study, we sought to take advantage of the COVID-19 pandemic to test this view. It is well-established that the surgical-type masks worn during the pandemic hinder the recognition and interpretation of facial emotion in typical participants. Relative to typical participants, we hypothesized that DPs may be disproportionately impaired when asked to interpret the facial emotion of people wearing face masks. We compared the ability of 34 DPs and 60 age-matched typical controls to recognize facial emotions i) when the whole face is visible, and ii) when the lower portion of the face is covered with a surgical mask. When expression stimuli were viewed without a mask, the DPs and typical controls exhibited similar levels of performance. However, when expression stimuli were shown with a mask, the DPs showed signs of subtle expression recognition deficits. The DPs were particularly prone to mislabeling masked expressions of happiness as emotion neutral. These results add to a growing body of evidence that under some conditions, DPs do exhibit subtle deficits of expression recognition.

How does the presence of a surgical face mask impair the perceived intensity of facial emotions?

The use of surgical-type face masks has become increasingly common during the COVID-19 pandemic. Recent findings suggest that it is harder to categorise the facial expressions of masked faces, than of unmasked faces. To date, studies of the effects of mask-wearing on emotion recognition have used categorisation paradigms: authors have presented facial expression stimuli and examined participants' ability to attach the correct label (e.g., happiness, disgust). While the ability to categorise particular expressions is important, this approach overlooks the fact that expression intensity is also informative during social interaction. For example, when predicting an interactant's future behaviour, it is useful to know whether they are slightly fearful or terrified, contented or very happy, slightly annoyed or angry. Moreover, because categorisation paradigms force observers to pick a single label to describe their percept, any additional dimensionality within observers' interpretation is lost. In the present study, we adopted a complementary emotion-intensity rating paradigm to study the effects of mask-wearing on expression interpretation. In an online experiment with 120 participants (82 female), we investigated how the presence of face masks affects the perceived emotional profile of prototypical expressions of happiness, sadness, anger, fear, disgust, and surprise. For each of these facial expressions, we measured the perceived intensity of all six emotions. We found that the perceived intensity of intended emotions (i.e., the emotion that the actor intended to convey) was reduced by the presence of a mask for all expressions except for anger. Additionally, when viewing all expressions except surprise, masks increased the perceived intensity of non-intended emotions (i.e., emotions that the actor did not intend to convey). Intensity ratings were unaffected by presentation duration (500ms vs 3000ms), or attitudes towards mask wearing. These findings shed light on the ambiguity that arises when interpreting the facial expressions of masked faces.

The Twenty Item Prosopagnosia Index (PI20) provides meaningful evidence of face recognition impairment.

The Twenty Item Prosopagnosia Index (PI20) is a self-report questionnaire used for quantifying prosopagnosic traits. This scale is intended to help researchers identify cases of developmental prosopagnosia by providing standardized self-report evidence to complement diagnostic evidence obtained from objective computer-based tasks. In order to respond appropriately to items, prosopagnosics must have some insight that their face recognition is well below average, while non-prosopagnosics need to understand that their relative face recognition ability falls within the typical range. There has been considerable debate about whether participants have the necessary insight into their face recognition abilities to respond appropriately. In the present study, we sought to determine whether the PI20 provides meaningful evidence of face recognition impairment. In keeping with the intended use of the instrument, we used PI20 scores to identify two groups: high-PI20 scorers (those with self-reported face recognition difficulties) and low-PI20 scorers (those with no self-reported face recognition difficulties). We found that participant groups distinguished on the basis of PI20 scores clearly differed in terms of their mean performance on objective measures of face recognition ability. We also found that high-PI20 scorers were more likely to achieve levels of face recognition accuracy associated with developmental prosopagnosia.

FFA and OFA Encode Distinct Types of Face Identity Information.

Faces of different people elicit distinct fMRI patterns in several face-selective regions of the human brain. Here we used representational similarity analysis to investigate what type of identity-distinguishing information is encoded in three face-selective regions: fusiform face area (FFA), occipital face area (OFA), and posterior superior temporal sulcus (pSTS). In a sample of 30 human participants (22 females, 8 males), we used fMRI to measure brain activity patterns elicited by naturalistic videos of famous face identities, and compared their representational distances in each region with models of the differences between identities. We built diverse candidate models, ranging from low-level image-computable properties (pixel-wise, GIST, and Gabor-Jet dissimilarities), through higher-level image-computable descriptions (OpenFace deep neural network, trained to cluster faces by identity), to complex human-rated properties (perceived similarity, social traits, and gender). We found marked differences in the information represented by the FFA and OFA. Dissimilarities between face identities in FFA were accounted for by differences in perceived similarity, Social Traits, Gender, and by the OpenFace network. In contrast, representational distances in OFA were mainly driven by differences in low-level image-based properties (pixel-wise and Gabor-Jet dissimilarities). Our results suggest that, although FFA and OFA can both discriminate between identities, the FFA representation is further removed from the image, encoding higher-level perceptual and social face information.SIGNIFICANCE STATEMENT Recent studies using fMRI have shown that several face-responsive brain regions can distinguish between different face identities. It is however unclear whether these different face-responsive regions distinguish between identities in similar or different ways. We used representational similarity analysis to investigate the computations within three brain regions in response to naturalistically varying videos of face identities. Our results revealed that two regions, the fusiform face area and the occipital face area, encode distinct identity information about faces. Although identity can be decoded from both regions, identity representations in fusiform face area primarily contained information about social traits, gender, and high-level visual features, whereas occipital face area primarily represented lower-level image features.

Normal recognition of famous voices in developmental prosopagnosia.

Developmental prosopagnosia (DP) is a condition characterised by lifelong face recognition difficulties. Recent neuroimaging findings suggest that DP may be associated with aberrant structure and function in multimodal regions of cortex implicated in the processing of both facial and vocal identity. These findings suggest that both facial and vocal recognition may be impaired in DP. To test this possibility, we compared the performance of 22 DPs and a group of typical controls, on closely matched tasks that assessed famous face and famous voice recognition ability. As expected, the DPs showed severe impairment on the face recognition task, relative to typical controls. In contrast, however, the DPs and controls identified a similar number of voices. Despite evidence of interactions between facial and vocal processing, these findings suggest some degree of dissociation between the two processing pathways, whereby one can be impaired while the other develops typically. A possible explanation for this dissociation in DP could be that the deficit originates in the early perceptual encoding of face structure, rather than at later, post-perceptual stages of face identity processing, which may be more likely to involve interactions with other modalities.

Holistic processing of facial identity in developmental prosopagnosia.

The nature of the perceptual deficit seen in developmental prosopagnosia remains poorly understood. One possibility is that these individuals experience face recognition difficulties because they fail to process faces holistically; they may be less able to analyze distal regions in parallel and therefore struggle to integrate information from different regions into a unified perceptual whole. Consequently, developmental prosopagnosics may be forced to base perceptual decisions on a slow, effortful piecemeal analysis of local facial features. In the present study, we sought to test this view by comparing the face recognition of developmental prosopagnosics and typical observers under two viewing conditions: when target faces were briefly presented in their entirety, and when they were inspected region-by-region through a dynamic aperture. If developmental prosopagnosics are forced to base perceptual decisions on information accumulated from a serial piecemeal analysis, one would expect little if any decrement in performance when target faces are viewed through apertures. Contrary to this prediction, however, developmental prosopagnosics showed strong aperture effects comparable with typical observers; their perceptual decisions were more accurate in the whole-face condition than when targets were viewed through the aperture. As expected, the developmental prosopagnosics were less accurate than typical controls when judging briefly presented faces shown in their entirety. Strikingly, however, they were also less able to accumulate perceptual evidence from a serial region-by-region analysis, than typical observers. Our results suggest that the perceptual problems seen in this population arise from imprecise descriptions of local regions, not aberrant holistic processing.

Faces and voices in the brain: A modality-general person-identity representation in superior temporal sulcus.

Face-selective and voice-selective brain regions have been shown to represent face-identity and voice-identity, respectively. Here we investigated whether there are modality-general person-identity representations in the brain that can be driven by either a face or a voice, and that invariantly represent naturalistically varying face videos and voice recordings of the same identity. Models of face and voice integration suggest that such representations could exist in multimodal brain regions, and in unimodal regions via direct coupling between face- and voice-selective regions. Therefore, in this study we used fMRI to measure brain activity patterns elicited by the faces and voices of familiar people in face-selective, voice-selective, and person-selective multimodal brain regions. We used representational similarity analysis to (1) compare representational geometries (i.e. representational dissimilarity matrices) of face- and voice-elicited identities, and to (2) investigate the degree to which pattern discriminants for pairs of identities generalise from one modality to the other. We did not find any evidence of similar representational geometries across modalities in any of our regions of interest. However, our results showed that pattern discriminants that were trained to discriminate pairs of identities from their faces could also discriminate the respective voices (and vice-versa) in the right posterior superior temporal sulcus (rpSTS). Our findings suggest that the rpSTS is a person-selective multimodal region that shows a modality-general person-identity representation and integrates face and voice identity information.

Low Vocal Pitch Preference Drives First Impressions Irrespective of Context in Male Voices but Not in Female Voices.

Vocal pitch has been found to influence judgments of perceived trustworthiness and dominance from a novel voice. However, the majority of findings arise from using only male voices and in context-specific scenarios. In two experiments, we first explore the influence of average vocal pitch on first-impression judgments of perceived trustworthiness and dominance, before establishing the existence of an overall preference for high or low pitch across genders. In Experiment 1, pairs of high- and low-pitched temporally reversed recordings of male and female vocal utterances were presented in a two-alternative forced-choice task. Results revealed a tendency to select the low-pitched voice over the high-pitched voice as more trustworthy, for both genders, and more dominant, for male voices only. Experiment 2 tested an overall preference for low-pitched voices, and whether judgments were modulated by speech content, using forward and reversed speech to manipulate context. Results revealed an overall preference for low pitch, irrespective of direction of speech, in male voices only. No such overall preference was found for female voices. We propose that an overall preference for low pitch is a default prior in male voices irrespective of context, whereas pitch preferences in female voices are more context- and situation-dependent. The present study confirms the important role of vocal pitch in the formation of first-impression personality judgments and advances understanding of the impact of context on pitch preferences across genders.