Face masks and fake masks: the effect of real and superimposed masks on face matching with super-recognisers, typical observers, and algorithms.

Mask wearing has been required in various settings since the outbreak of COVID-19, and research has shown that identity judgements are difficult for faces wearing masks. To date, however, the majority of experiments on face identification with masked faces tested humans and computer algorithms using images with superimposed masks rather than images of people wearing real face coverings. In three experiments we test humans (control participants and super-recognisers) and algorithms with images showing different types of face coverings. In all experiments we tested matching concealed or unconcealed faces to an unconcealed reference image, and we found a consistent decrease in face matching accuracy with masked compared to unconcealed faces. In Experiment 1, typical human observers were most accurate at face matching with unconcealed images, and poorer for three different types of superimposed mask conditions. In Experiment 2, we tested both typical observers and super-recognisers with superimposed and real face masks, and found that performance was poorer for real compared to superimposed masks. The same pattern was observed in Experiment 3 with algorithms. Our results highlight the importance of testing both humans and algorithms with real face masks, as using only superimposed masks may underestimate their detrimental effect on face identification.

The effect of face masks and sunglasses on identity and expression recognition with super-recognizers and typical observers.

Face masks present a new challenge to face identification (here matching) and emotion recognition in Western cultures. Here, we present the results of three experiments that test the effect of masks, and also the effect of sunglasses (an occlusion that individuals tend to have more experienced with) on (i) familiar face matching, (ii) unfamiliar face matching and (iii) emotion categorization. Occlusion reduced accuracy in all three tasks, with most errors in the mask condition; however, there was little difference in performance for faces in masks compared with faces in sunglasses. Super-recognizers, people who are highly skilled at matching unconcealed faces, were impaired by occlusion, but at the group level, performed with higher accuracy than controls on all tasks. Results inform psychology theory with implications for everyday interactions, security and policing in a mask-wearing society.

Seeing through disguise: Getting to know you with a deep convolutional neural network.

People use disguise to look unlike themselves (evasion) or to look like someone else (impersonation). Evasion disguise challenges human ability to see an identity across variable images; Impersonation challenges human ability to tell people apart. Personal familiarity with an individual face helps humans to see through disguise. Here we propose a model of familiarity based on high-level visual learning mechanisms that we tested using a deep convolutional neural network (DCNN) trained for face identification. DCNNs generate a face space in which identities and images co-exist in a unified computational framework, that is categorically structured around identity, rather than retinotopy. This allows for simultaneous manipulation of mechanisms that contrast identities and cluster images. In Experiment 1, we measured the DCNN's baseline accuracy (unfamiliar condition) for identification of faces in no disguise and disguise conditions. Disguise affected DCNN performance in much the same way it affects human performance for unfamiliar faces in disguise (cf. Noyes & Jenkins, 2019). In Experiment 2, we simulated familiarity for individual identities by averaging the DCNN-generated representations from multiple images of each identity. Averaging improved DCNN recognition of faces in evasion disguise, but reduced the ability of the DCNN to differentiate identities of similar appearance. In Experiment 3, we implemented a contrast learning technique to simultaneously teach the DCNN appearance variation and identity contrasts between different individuals. This facilitated identification with both evasion and impersonation disguise. Familiar face recognition requires an ability to group images of the same identity together and separate different identities. The deep network provides a high-level visual representation for face recognition that supports both of these mechanisms of face learning simultaneously.

We need to talk about super-recognizers Invited commentary on: Ramon, M., Bobak, A. K., & White, D. Super-recognizers: From the lab to the world and back again. British Journal of Psychology.

Whilst we agree with much of what Ramon et al. (2019, British Journal of Psychology) say, we emphasize the additional importance of taking into account the often-neglected psychometric properties of existing and future techniques.

Deliberate disguise in face identification.

Facial image comparison is difficult for unfamiliar faces and easy for familiar faces. Those conclusions are robust, but they arise from situations in which the people being identified cooperate with the effort to identify them. In forensic and security settings, people are often motivated to subvert identification by manipulating their appearance, yet little is known about deliberate disguise and its effectiveness. We distinguish two forms of disguise-Evasion (trying not to look like oneself) and Impersonation (trying to look like another person). We present a new set of disguised face images (the FAÇADE image set), in which models altered their appearance to induce specific identification errors. In Experiment 1, unfamiliar observers were less accurate matching disguise items, especially evasion items, than matching undisguised items. A similar pattern held in Experiment 2, in which participants were informed about the disguise manipulations. In Experiment 3, familiar observers saw through impersonation disguise, but accuracy was lower for evasion disguise. Quantifying the performance cost of disguise reveals distinct performance profiles for impersonation and evasion. Evasion disguise was especially effective and reduced identification performance for familiar observers as well as for unfamiliar observers. We subsume these findings under a statistical framework of face learning. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Learning context and the other-race effect: Strategies for improving face recognition.

People recognize faces of their own race more accurately than faces of other races-a phenomenon known as the "Other-Race Effect" (ORE). Previous studies show that training with multiple variable images improves face recognition. Building on multi-image training, we take a novel approach to improving own- and other-race face recognition by testing the role of learning context on accuracy. Learning context was either contiguous, with multiple images of each identity seen in sequence, or distributed, with multiple images of an identity randomly interspersed among different identities. In two experiments, East Asian and Caucasian participants learned own- and other-races faces either in a contiguous or distributed order. In Experiment 1, people learned each identity from four highly variable face images. In Experiment 2, identities were learned from one image, repeated four times. In both experiments we found a robust other-race effect. The effect of learning context, however, differed depending on the variability of the learned images. The distributed presentation yielded better recognition when people learned from single repeated images (Exp. 1), but not when they learned from multiple variable images (Exp. 2). Overall, performance was better with multiple-image training than repeated single image training. We conclude that multiple-image training and distributed learning can both improve recognition accuracy, but via distinct processes. The former broadens perceptual tolerance for image variation from a face, when there are diverse images available to learn. The latter effectively strengthens the representation of differences among similar faces, when there is only a single learning image.

Face recognition ability does not predict person identification performance: using individual data in the interpretation of group results.

There are large individual differences in people's face recognition ability. These individual differences provide an opportunity to recruit the best face-recognisers into jobs that require accurate person identification, through the implementation of ability-screening tasks. To date, screening has focused exclusively on face recognition ability; however real-world identifications can involve the use of other person-recognition cues. Here we incorporate body and biological motion recognition as relevant skills for person identification. We test whether performance on a standardised face-matching task (the Glasgow Face Matching Test) predicts performance on three other identity-matching tasks, based on faces, bodies, and biological motion. We examine the results from group versus individual analyses. We found stark differences between the conclusions one would make from group analyses versus analyses that retain information about individual differences. Specifically, tests of correlation and analysis of variance suggested that face recognition ability was related to performance for all person identification tasks. These analyses were strikingly inconsistent with the individual differences data, which suggested that the screening task was related only to performance on the face task. This study highlights the importance of individual data in the interpretation of results of person identification ability.

Face recognition accuracy of forensic examiners, superrecognizers, and face recognition algorithms.

Achieving the upper limits of face identification accuracy in forensic applications can minimize errors that have profound social and personal consequences. Although forensic examiners identify faces in these applications, systematic tests of their accuracy are rare. How can we achieve the most accurate face identification: using people and/or machines working alone or in collaboration? In a comprehensive comparison of face identification by humans and computers, we found that forensic facial examiners, facial reviewers, and superrecognizers were more accurate than fingerprint examiners and students on a challenging face identification test. Individual performance on the test varied widely. On the same test, four deep convolutional neural networks (DCNNs), developed between 2015 and 2017, identified faces within the range of human accuracy. Accuracy of the algorithms increased steadily over time, with the most recent DCNN scoring above the median of the forensic facial examiners. Using crowd-sourcing methods, we fused the judgments of multiple forensic facial examiners by averaging their rating-based identity judgments. Accuracy was substantially better for fused judgments than for individuals working alone. Fusion also served to stabilize performance, boosting the scores of lower-performing individuals and decreasing variability. Single forensic facial examiners fused with the best algorithm were more accurate than the combination of two examiners. Therefore, collaboration among humans and between humans and machines offers tangible benefits to face identification accuracy in important applications. These results offer an evidence-based roadmap for achieving the most accurate face identification possible.

Wisdom of the social versus non-social crowd in face identification.

Face identification is more accurate when people collaborate in social dyads than when they work alone (Dowsett & Burton, 2015, Br. J. Psychol., 106, 433). Identification accuracy is also increased when the responses of two people are averaged for each item to create a 'non-social' dyad (White, Burton, Kemp, & Jenkins, 2013, Appl. Cogn. Psychol., 27, 769; White et al., 2015, Proc. R. Soc. B Biol. Sci., 282, 20151292). Does social collaboration add to the benefits of response averaging for face identification? We compared individuals, social dyads, and non-social dyads on an unfamiliar face identity-matching test. We also simulated non-social collaborations for larger groups of people. Individuals and social dyads judged whether face image pairs depicted the same- or different identities, responding on a 5-point certainty scale. Non-social dyads were constructed by averaging the responses of paired individuals. Both social and non-social dyads were more accurate than individuals. There was no advantage for social over non-social dyads. For larger non-social groups, performance peaked at near perfection with a crowd size of eight participants. We tested three computational models of social collaboration and found that social dyad performance was predicted by the decision of the more accurate partner. We conclude that social interaction does not bolster accuracy for unfamiliar face identity matching in dyads beyond what can be achieved by averaging judgements.

Hyper-realistic face masks: a new challenge in person identification.

We often identify people using face images. This is true in occupational settings such as passport control as well as in everyday social environments. Mapping between images and identities assumes that facial appearance is stable within certain bounds. For example, a person's apparent age, gender and ethnicity change slowly, if at all. It also assumes that deliberate changes beyond these bounds (i.e., disguises) would be easy to spot. Hyper-realistic face masks overturn these assumptions by allowing the wearer to look like an entirely different person. If unnoticed, these masks break the link between facial appearance and personal identity, with clear implications for applied face recognition. However, to date, no one has assessed the realism of these masks, or specified conditions under which they may be accepted as real faces. Herein, we examined incidental detection of unexpected but attended hyper-realistic masks in both photographic and live presentations. Experiment 1 (UK; n = 60) revealed no evidence for overt detection of hyper-realistic masks among real face photos, and little evidence of covert detection. Experiment 2 (Japan; n = 60) extended these findings to different masks, mask-wearers and participant pools. In Experiment 3 (UK and Japan; n = 407), passers-by failed to notice that a live confederate was wearing a hyper-realistic mask and showed limited evidence of covert detection, even at close viewing distance (5 vs. 20 m). Across all of these studies, viewers accepted hyper-realistic masks as real faces. Specific countermeasures will be required if detection rates are to be improved.

Camera-to-subject distance affects face configuration and perceived identity.

Face identification is reliable for viewers who are familiar with the face, and unreliable for viewers who are not. One account of this contrast is that people become good at recognising a face by learning its configuration-the specific pattern of feature-to-feature measurements. In practice, these measurements differ across photos of the same face because objects appear more flat or convex depending on their distance from the camera. Here we connect this optical understanding to face configuration and identification accuracy. Changing camera-to-subject distance (0.32m versus 2.70m) impaired perceptual matching of unfamiliar faces, even though the images were presented at the same size. Familiar face matching was accurate across conditions. Reinstating valid distance cues mitigated the performance cost, suggesting that perceptual constancy compensates for distance-related changes in optical face shape. Acknowledging these distance effects could reduce identification errors in applied settings such as passport control.

Face Recognition by Metropolitan Police Super-Recognisers.

Face recognition is used to prove identity across a wide variety of settings. Despite this, research consistently shows that people are typically rather poor at matching faces to photos. Some professional groups, such as police and passport officers, have been shown to perform just as poorly as the general public on standard tests of face recognition. However, face recognition skills are subject to wide individual variation, with some people showing exceptional ability-a group that has come to be known as 'super-recognisers'. The Metropolitan Police Force (London) recruits 'super-recognisers' from within its ranks, for deployment on various identification tasks. Here we test four working super-recognisers from within this police force, and ask whether they are really able to perform at levels above control groups. We consistently find that the police 'super-recognisers' perform at well above normal levels on tests of unfamiliar and familiar face matching, with degraded as well as high quality images. Recruiting employees with high levels of skill in these areas, and allocating them to relevant tasks, is an efficient way to overcome some of the known difficulties associated with unfamiliar face recognition.