Understanding 'error' in the forensic sciences: A primer.

This paper distils seven key lessons about 'error' from a collaborative webinar series between practitioners at Victoria Police Forensic Services Department and academics. It aims to provide the common understanding of error necessary to foster interdisciplinary dialogue, collaboration and research. The lessons underscore the inevitability, complexity and subjectivity of error, as well as opportunities for learning and growth. Ultimately, we argue that error can be a potent tool for continuous improvement and accountability, enhancing the reliability of forensic sciences and public trust. It is hoped the shared understanding provided by this paper will support future initiatives and funding for collaborative developments in this vital domain.

The novel object-matching test (NOM Test): A psychometric measure of visual comparison ability.

This paper presents a new test of object-matching ability: the Novel Object-Matching Test (NOM Test). Object-matching (or visual comparison) is a complex cognitive and perceptual visual comparison task undertaken by forensic scientists - yet no openly available, standardised and psychometrically validated test of object-matching ability exists. This is in contrast to other visual comparison domains like face-matching where many tests are widely available. In this paper, we present the development and psychometric validation of the first openly available object-matching test where people view two complex artificial visual patterns side-by-side and decide if they are from the same source or different sources. We provide normative data and psychometric properties for two long-form and two short-form versions of the test, and two additional versions designed to identify high and low-performers. We also provide evidence of discriminant validity and convergent validity that demonstrates the NOM Test correlates strongly with other object-matching tasks like fingerprint-matching - but not other tasks requiring cognitive-perceptual skill (e.g., visual intelligence). The NOM Test is free for research use with acknowledgment and is available at https://osf.io/pv6ye/ .

Diverse types of expertise in facial recognition.

Facial recognition errors can jeopardize national security, criminal justice, public safety and civil rights. Here, we compare the most accurate humans and facial recognition technology in a detailed lab-based evaluation and international proficiency test for forensic scientists involving 27 forensic departments from 14 countries. We find striking cognitive and perceptual diversity between naturally skilled super-recognizers, trained forensic examiners and deep neural networks, despite them achieving equivalent accuracy. Clear differences emerged in super-recognizers' and forensic examiners' perceptual processing, errors, and response patterns: super-recognizers were fast, biased to respond 'same person' and misidentified people with extreme confidence, whereas forensic examiners were slow, unbiased and strategically avoided misidentification errors. Further, these human experts and deep neural networks disagreed on the similarity of faces, pointing to differences in their representations of faces. Our findings therefore reveal multiple types of facial recognition expertise, with each type lending itself to particular facial recognition roles in operational settings. Finally, we show that harnessing the diversity between individual experts provides a robust method of maximizing facial recognition accuracy. This can be achieved either via collaboration between experts in forensic laboratories, or most promisingly, by statistical fusion of match scores provided by different types of expert.

Selecting police super-recognisers.

People vary in their ability to recognise faces. These individual differences are consistent over time, heritable and associated with brain anatomy. This implies that face identity processing can be improved in applied settings by selecting high performers-'super-recognisers' (SRs)-but these selection processes are rarely available for scientific scrutiny. Here we report an 'end-to-end' selection process used to establish an SR 'unit' in a large police force. Australian police officers (n = 1600) completed 3 standardised face identification tests and we recruited 38 SRs from this cohort to complete 10 follow-up tests. As a group, SRs were 20% better than controls in lab-based tests of face memory and matching, and equalled or surpassed accuracy of forensic specialists that currently perform face identification tasks for police. Individually, SR accuracy was variable but this problem was mitigated by adopting strict selection criteria. SRs' superior abilities transferred only partially to body identity decisions where the face was not visible, and they were no better than controls at deciding which visual scene that faces had initially been encountered in. Notwithstanding these important qualifications, we conclude that super-recognisers are an effective solution to improving face identity processing in applied settings.

Looking at faces in the wild.

Faces are key to everyday social interactions, but our understanding of social attention is based on experiments that present images of faces on computer screens. Advances in wearable eye-tracking devices now enable studies in unconstrained natural settings but this approach has been limited by manual coding of fixations. Here we introduce an automatic 'dynamic region of interest' approach that registers eye-fixations to bodies and faces seen while a participant moves through the environment. We show that just 14% of fixations are to faces of passersby, contrasting with prior screen-based studies that suggest faces automatically capture visual attention. We also demonstrate the potential for this new tool to help understand differences in individuals' social attention, and the content of their perceptual exposure to other people. Together, this can form the basis of a new paradigm for studying social attention 'in the wild' that opens new avenues for theoretical, applied and clinical research.

Let's talk about faces: Identifying faces from verbal descriptions.

Face descriptions inform real-world identification decisions, for example when eyewitnesses describe criminal perpetrators. However, it is unclear how effective face descriptions are for identification. Here, we examined the accuracy of face identification from verbal descriptions, and how individual differences in face perception relate to producing and using descriptions for identification. In Study 1, participants completed a face communication task in pairs. Each participant saw a single face, and via verbal communication only, the pair decided if they were viewing the same person or different people. Dyads achieved 72% accuracy, compared to 81% when participants completed the task individually by matching face pairs side-by-side. Performance on the face communication and perceptual matching tasks were uncorrelated, perhaps due to low measurement reliability of the face communication task. In subsequent studies, we examined the abilities of face 'describers' (Study 2) and 'identifiers' separately (Study 3). We found that 'super-recognizers' - people with extremely high perceptual face identification abilities - outperformed controls in both studies. Overall, these results show that people can successfully describe faces for identification. Preliminary evidence suggests that this ability - and the ability use facial descriptions for identification - has some association with perceptual face identification skill.

The low prevalence effect in fingerprint comparison amongst forensic science trainees and novices.

The low prevalence effect is a phenomenon whereby target prevalence affects performance in visual search (e.g., baggage screening) and comparison (e.g., fingerprint examination) tasks, such that people more often fail to detect infrequent target stimuli. For example, when exposed to higher base-rates of 'matching' (i.e., from the same person) than 'non-matching' (i.e., from different people) fingerprint pairs, people more often misjudge 'non-matching' pairs as 'matches'-an error that can falsely implicate an innocent person for a crime they did not commit. In this paper, we investigated whether forensic science training may mitigate the low prevalence effect in fingerprint comparison. Forensic science trainees (n = 111) and untrained novices (n = 114) judged 100 fingerprint pairs as 'matches' or 'non-matches' where the matching pair occurrence was either high (90%) or equal (50%). Some participants were also asked to use a novel feature-comparison strategy as a potential attenuation technique for the low prevalence effect. Regardless of strategy, both trainees and novices were susceptible to the effect, such that they more often misjudged non-matching pairs as matches when non-matches were rare. These results support the robust nature of the low prevalence effect in visual comparison and have important applied implications for forensic decision-making in the criminal justice system.

Statistical feature training improves fingerprint-matching accuracy in novices and professional fingerprint examiners.

Forensic science practitioners compare visual evidence samples (e.g. fingerprints) and decide if they originate from the same person or different people (i.e. fingerprint 'matching'). These tasks are perceptually and cognitively complex-even practising professionals can make errors-and what limited research exists suggests that existing professional training is ineffective. This paper presents three experiments that demonstrate the benefit of perceptual training derived from mathematical theories that suggest statistically rare features have diagnostic utility in visual comparison tasks. Across three studies (N = 551), we demonstrate that a brief module training participants to focus on statistically rare fingerprint features improves fingerprint-matching performance in both novices and experienced fingerprint examiners. These results have applied importance for improving the professional performance of practising fingerprint examiners, and even other domains where this technique may also be helpful (e.g. radiology or banknote security).

Masked face identification is improved by diagnostic feature training.

To slow the spread of COVID-19, many people now wear face masks in public. Face masks impair our ability to identify faces, which can cause problems for professional staff who identify offenders or members of the public. Here, we investigate whether performance on a masked face matching task can be improved by training participants to compare diagnostic facial features (the ears and facial marks)-a validated training method that improves matching performance for unmasked faces. We show this brief diagnostic feature training, which takes less than two minutes to complete, improves matching performance for masked faces by approximately 5%. A control training course, which was unrelated to face identification, had no effect on matching performance. Our findings demonstrate that comparing the ears and facial marks is an effective means of improving face matching performance for masked faces. These findings have implications for professions that regularly perform face identification.

Match me if you can: Evidence for a domain-general visual comparison ability.

Visual comparison-comparing visual stimuli (e.g., fingerprints) side by side and determining whether they originate from the same or different source (i.e., "match")-is a complex discrimination task involving many cognitive and perceptual processes. Despite the real-world consequences of this task, which is often conducted by forensic scientists, little is understood about the psychological processes underpinning this ability. There are substantial individual differences in visual comparison accuracy amongst both professionals and novices. The source of this variation is unknown, but may reflect a domain-general and naturally varying perceptual ability. Here, we investigate this by comparing individual differences (N = 248 across two studies) in four visual comparison domains: faces, fingerprints, firearms, and artificial prints. Accuracy on all comparison tasks was significantly correlated and accounted for a substantial portion of variance (e.g., 42% in Exp. 1) in performance across all tasks. Importantly, this relationship cannot be attributed to participants' intrinsic motivation or skill in other visual-perceptual tasks (visual search and visual statistical learning). This paper provides novel evidence of a reliable, domain-general visual comparison ability.

Diagnostic feature training improves face matching accuracy.

Identifying unfamiliar faces is surprisingly error-prone, even for experienced professionals who perform this task regularly. Previous attempts to train this ability have been largely unsuccessful, leading many to conclude that face identity processing is hard-wired and not amenable to further perceptual learning. Here, we take a novel expert knowledge elicitation approach to training, based on the feature-based comparison strategy used by high-performing professional facial examiners. We show that instructing novices to focus on the facial features that are most diagnostic of identity for these experts-the ears and facial marks (e.g., scars, freckles and blemishes)-improves accuracy on unfamiliar face matching tasks by 6%. This training takes just 6 min to complete and yet accounts for approximately half of experts' superiority on the task. Benefits of training are strongest when diagnostic features are clearly visible and absent when participants are trained to rely on nondiagnostic features. Our data-driven approach contrasts with theory-driven training that is designed to improve holistic face processing mechanisms associated with familiar face recognition. This suggests that protocols which bypass the core face recognition system-and instead reorient attention to features that are undervalued by novices-offer a more promising route to training for unfamiliar face matching. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

UNSW Face Test: A screening tool for super-recognizers.

We present a new test-the UNSW Face Test (www.unswfacetest.com)-that has been specifically designed to screen for super-recognizers in large online cohorts and is available free for scientific use. Super-recognizers are people that demonstrate sustained performance in the very top percentiles in tests of face identification ability. Because they represent a small proportion of the population, screening large online cohorts is an important step in their initial recruitment, before confirmatory testing via standardized measures and more detailed cognitive testing. We provide normative data on the UNSW Face Test from 3 cohorts tested via the internet (combined n = 23,902) and 2 cohorts tested in our lab (combined n = 182). The UNSW Face Test: (i) captures both identification memory and perceptual matching, as confirmed by correlations with existing tests of these abilities; (ii) captures face-specific perceptual and memorial abilities, as confirmed by non-significant correlations with non-face object processing tasks; (iii) enables researchers to apply stricter selection criteria than other available tests, which boosts the average accuracy of the individuals selected in subsequent testing. Together, these properties make the test uniquely suited to screening for super-recognizers in large online cohorts.

Do professional facial image comparison training courses work?

Facial image comparison practitioners compare images of unfamiliar faces and decide whether or not they show the same person. Given the importance of these decisions for national security and criminal investigations, practitioners attend training courses to improve their face identification ability. However, these courses have not been empirically validated so it is unknown if they improve accuracy. Here, we review the content of eleven professional training courses offered to staff at national security, police, intelligence, passport issuance, immigration and border control agencies around the world. All reviewed courses include basic training in facial anatomy and prescribe facial feature (or 'morphological') comparison. Next, we evaluate the effectiveness of four representative courses by comparing face identification accuracy before and after training in novices (n = 152) and practitioners (n = 236). We find very strong evidence that short (1-hour and half-day) professional training courses do not improve identification accuracy, despite 93% of trainees believing their performance had improved. We find some evidence of improvement in a 3-day training course designed to introduce trainees to the unique feature-by-feature comparison strategy used by facial examiners in forensic settings. However, observed improvements are small, inconsistent across tests, and training did not produce the qualitative changes associated with examiners' expertise. Future research should test the benefits of longer examination-focussed training courses and incorporate longitudinal approaches to track improvements caused by mentoring and deliberate practice. In the absence of evidence that training is effective, we advise agencies to explore alternative evidence-based strategies for improving the accuracy of face identification decisions.

Familiarity and Within-Person Facial Variability: The Importance of the Internal and External Features.

As faces become familiar, we come to rely more on their internal features for recognition and matching tasks. Here, we assess whether this same pattern is also observed for a card sorting task. Participants sorted photos showing either the full face, only the internal features, or only the external features into multiple piles, one pile per identity. In Experiments 1 and 2, we showed the standard advantage for familiar faces-sorting was more accurate and showed very few errors in comparison with unfamiliar faces. However, for both familiar and unfamiliar faces, sorting was less accurate for external features and equivalent for internal and full faces. In Experiment 3, we asked whether external features can ever be used to make an accurate sort. Using familiar faces and instructions on the number of identities present, we nevertheless found worse performance for the external in comparison with the internal features, suggesting that less identity information was available in the former. Taken together, we show that full faces and internal features are similarly informative with regard to identity. In comparison, external features contain less identity information and produce worse card sorting performance. This research extends current thinking on the shift in focus, both in attention and importance, toward the internal features and away from the external features as familiarity with a face increases.

Thinking forensics: Cognitive science for forensic practitioners.

Human factors and their implications for forensic science have attracted increasing levels of interest across criminal justice communities in recent years. Initial interest centred on cognitive biases, but has since expanded such that knowledge from psychology and cognitive science is slowly infiltrating forensic practices more broadly. This article highlights a series of important findings and insights of relevance to forensic practitioners. These include research on human perception, memory, context information, expertise, decision-making, communication, experience, verification, confidence, and feedback. The aim of this article is to sensitise forensic practitioners (and lawyers and judges) to a range of potentially significant issues, and encourage them to engage with research in these domains so that they may adapt procedures to improve performance, mitigate risks and reduce errors. Doing so will reduce the divide between forensic practitioners and research scientists as well as improve the value and utility of forensic science evidence.

Evaluating the feature comparison strategy for forensic face identification.

Face recognition is thought to rely on representations that encode holistic properties. Paradoxically, professional forensic examiners who identify unfamiliar faces by comparing facial images are trained to adopt a feature-by-feature comparison strategy. Here we tested the effectiveness of this strategy by asking participants to rate facial feature similarity prior to making same/different identity decisions to pairs of face images. Experiment 1 provided preliminary evidence that rating feature similarity improves unfamiliar face matching accuracy in novice participants. In Experiment 2, we found benefits of this procedure over and above rating similarity of personality traits and image quality parameters, suggesting that benefits are not solely attributable to general increases in attention. In Experiment 3, we then compared performance of trained forensic facial image examiners to novice participants, and found that examiners displayed: i) superior face matching accuracy; ii) smaller face inversion and feature inversion effects; and iii) feature ratings that were more diagnostic of identity. Further, aggregating feature ratings of multiple examiners produced perfect identity discrimination. Based on these quantitative and qualitative differences between experts and novices, we conclude that comparison based on local features confers specific benefits to trained forensic examiners. (PsycINFO Database Record

Evaluating training methods for facial image comparison: the face shape strategy does not work.

Human performance on unfamiliar face matching is known to be highly error prone. However, in organisations where staff are required to perform this task as part of their daily work, attempts are often made to mitigate risk by providing training. Importantly, the methods used in these training courses have not been subjected to empirical validation. In this study we evaluate a common component of many training programmes which encourages viewers to classify face shape. Our results show very low agreement in face shape classification, both within and between participants, and across repeated presentations of a single image to a single participant. Furthermore, face shape classification training did not improve face matching accuracy, suggesting that the face shape strategy does not facilitate identification.