Trust in automation and the accuracy of human-algorithm teams performing one-to-one face matching tasks.

The human face is commonly used for identity verification. While this task was once exclusively performed by humans, technological advancements have seen automated facial recognition systems (AFRS) integrated into many identification scenarios. Although many state-of-the-art AFRS are exceptionally accurate, they often require human oversight or involvement, such that a human operator actions the final decision. Previously, we have shown that on average, humans assisted by a simulated AFRS (sAFRS) failed to reach the level of accuracy achieved by the same sAFRS alone, due to overturning the system's correct decisions and/or failing to correct sAFRS errors. The aim of the current study was to investigate whether participants' trust in automation was related to their performance on a one-to-one face matching task when assisted by a sAFRS. Participants (n = 160) completed a standard face matching task in two phases: an unassisted baseline phase, and an assisted phase where they were shown the identification decision (95% accurate) made by a sAFRS prior to submitting their own decision. While most participants improved with sAFRS assistance, those with greater relative trust in automation achieved larger gains in performance. However, the average aided performance of participants still failed to reach that of the sAFRS alone, regardless of trust status. Nonetheless, further analysis revealed a small sample of participants who achieved 100% accuracy when aided by the sAFRS. Our results speak to the importance of considering individual differences when selecting employees for roles requiring human-algorithm interaction, including identity verification tasks that incorporate facial recognition technologies.

Face masks and fake masks: the effect of real and superimposed masks on face matching with super-recognisers, typical observers, and algorithms.

Mask wearing has been required in various settings since the outbreak of COVID-19, and research has shown that identity judgements are difficult for faces wearing masks. To date, however, the majority of experiments on face identification with masked faces tested humans and computer algorithms using images with superimposed masks rather than images of people wearing real face coverings. In three experiments we test humans (control participants and super-recognisers) and algorithms with images showing different types of face coverings. In all experiments we tested matching concealed or unconcealed faces to an unconcealed reference image, and we found a consistent decrease in face matching accuracy with masked compared to unconcealed faces. In Experiment 1, typical human observers were most accurate at face matching with unconcealed images, and poorer for three different types of superimposed mask conditions. In Experiment 2, we tested both typical observers and super-recognisers with superimposed and real face masks, and found that performance was poorer for real compared to superimposed masks. The same pattern was observed in Experiment 3 with algorithms. Our results highlight the importance of testing both humans and algorithms with real face masks, as using only superimposed masks may underestimate their detrimental effect on face identification.

Simulated automated facial recognition systems as decision-aids in forensic face matching tasks.

Automated Facial Recognition Systems (AFRS) are used by governments, law enforcement agencies, and private businesses to verify the identity of individuals. Although previous research has compared the performance of AFRS and humans on tasks of one-to-one face matching, little is known about how effectively human operators can use these AFRS as decision-aids. Our aim was to investigate how the prior decision from an AFRS affects human performance on a face matching task, and to establish whether human oversight of AFRS decisions can lead to collaborative performance gains for the human-algorithm team. The identification decisions from our simulated AFRS were informed by the performance of a real, state-of-the-art, Deep Convolutional Neural Network (DCNN) AFRS on the same task. Across five pre-registered experiments, human operators used the decisions from highly accurate AFRS (> 90%) to improve their own face matching performance compared with baseline (sensitivity gain: Cohen's d = 0.71-1.28; overall accuracy gain: d = 0.73-1.46). Yet, despite this improvement, AFRS-aided human performance consistently failed to reach the level that the AFRS achieved alone. Even when the AFRS erred only on the face pairs with the highest human accuracy (> 89%), participants often failed to correct the system's errors, while also overruling many correct decisions, raising questions about the conditions under which human oversight might enhance AFRS operation. Overall, these data demonstrate that the human operator is a limiting factor in this simple model of human-AFRS teaming. These findings have implications for the "human-in-the-loop" approach to AFRS oversight in forensic face matching scenarios. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Masked face identification is improved by diagnostic feature training.

To slow the spread of COVID-19, many people now wear face masks in public. Face masks impair our ability to identify faces, which can cause problems for professional staff who identify offenders or members of the public. Here, we investigate whether performance on a masked face matching task can be improved by training participants to compare diagnostic facial features (the ears and facial marks)-a validated training method that improves matching performance for unmasked faces. We show this brief diagnostic feature training, which takes less than two minutes to complete, improves matching performance for masked faces by approximately 5%. A control training course, which was unrelated to face identification, had no effect on matching performance. Our findings demonstrate that comparing the ears and facial marks is an effective means of improving face matching performance for masked faces. These findings have implications for professions that regularly perform face identification.

The effect of cognitive load on horizontal and vertical spatial asymmetries.

Healthy individuals typically show a leftward attentional bias in the allocation of spatial attention along the horizontal plane, a phenomenon known as pseudoneglect, which relies on a right hemispheric dominance for visuospatial processing. Also, healthy individuals tend to overestimate the upper hemispace when orienting attention along the vertical plane, a phenomenon that may depend on asymmetric ventral and dorsal visual streams activation. Previous research has demonstrated that when attentional resources are reduced due to increased cognitive load, pseudoneglect is attenuated (or even reversed), due to decreased right-hemispheric activations. Critically, whether and how the reduction of attentional resources under load modulates vertical spatial asymmetries has not been addressed before. We asked participants to perform a line bisection task both with and without the addition of a concurrent auditory working memory task with lines oriented either horizontally or vertically. Results showed that increasing cognitive load reduced the typical leftward/upward bias with no difference between orientations. Our data suggest that the degree of cognitive load affects spatial attention not only in the horizontal but also in the vertical plane. Lastly, the similar effect of load on horizontal and vertical judgements suggests these biases may be related to only partially independent mechanisms.

The dissociable influence of social context on judgements of facial attractiveness and trustworthiness.

The cheerleader effect occurs when the same face is rated to be more attractive when it is seen in a group compared to when seen alone. We investigated whether this phenomenon also occurs for trustworthiness judgements, and examined how these effects are influenced by the characteristics of the individual being evaluated and those of the group they are seen in. Across three experiments, we reliably replicated the cheerleader effect. Most faces became more attractive in a group. Yet, the size of the cheerleader effect that each face experienced was not related to its own attractiveness, nor to the attractiveness of the group or the group's digitally averaged face. We discuss the implications of our findings for the hierarchical encoding and contrast mechanisms that have previously been used to explain the cheerleader effect. Surprisingly, judgements of facial trustworthiness did not experience a 'cheerleader effect'. Instead, we found that untrustworthy faces became significantly more trustworthy in all groups, while there was no change for faces that were already trustworthy alone. Taken together, our results demonstrate that social context can have a dissociable influence on our first impressions, depending on the trait being evaluated.

The "cheerleader effect" in facial and bodily attractiveness: A result of memory bias and not perceptual encoding.

Individual faces are rated as more attractive when presented in a group compared with when presented individually; a finding dubbed the "cheerleader effect." As a relatively recent discovery, the conditions necessary to observe the effect are not clearly understood. We sought to better define these conditions by examining two parameters associated with the effect. Our first aim was to determine whether the effect is specific to faces or occurs also for human bodies. Both face and body images were rated as being more attractive when presented in groups than when presented in isolation, demonstrating that the cheerleader effect is not restricted to faces. Furthermore, the effect was significantly larger for bodies than faces. Our second aim was to determine whether the cheerleader effect originates from a bias in memory or occurs during perceptual encoding. Participants in the "memory" condition provided attractiveness ratings after images had been removed from the testing screen, whereas participants in the "perceptual" condition provided ratings while the images remained visible, thereby eliminating the memory components of the paradigm. Significant cheerleader effects were only observed in the memory condition. We conclude that the cheerleader effect for faces and bodies is due to a bias in memory and does not occur at an initial stage of perceptual encoding.

Surgical face masks impair human face matching performance for familiar and unfamiliar faces.

In response to the COVID-19 pandemic, many governments around the world now recommend, or require, that their citizens cover the lower half of their face in public. Consequently, many people now wear surgical face masks in public. We investigated whether surgical face masks affected the performance of human observers, and a state-of-the-art face recognition system, on tasks of perceptual face matching. Participants judged whether two simultaneously presented face photographs showed the same person or two different people. We superimposed images of surgical masks over the faces, creating three different mask conditions: control (no masks), mixed (one face wearing a mask), and masked (both faces wearing masks). We found that surgical face masks have a large detrimental effect on human face matching performance, and that the degree of impairment is the same regardless of whether one or both faces in each pair are masked. Surprisingly, this impairment is similar in size for both familiar and unfamiliar faces. When matching masked faces, human observers are biased to reject unfamiliar faces as "mismatches" and to accept familiar faces as "matches". Finally, the face recognition system showed very high classification accuracy for control and masked stimuli, even though it had not been trained to recognise masked faces. However, accuracy fell markedly when one face was masked and the other was not. Our findings demonstrate that surgical face masks impair the ability of humans, and naïve face recognition systems, to perform perceptual face matching tasks. Identification decisions for masked faces should be treated with caution.

Limited evidence of hierarchical encoding in the cheerleader effect.

"The cheerleader effect" refers to the increase in attractiveness that an individual face experiences when seen in a group of other faces. It has been proposed that the cheerleader effect occurs because (a) the visual system rapidly summarises a group of faces into an ensemble representation, (b) which is hypothesised to be highly attractive because of its average facial characteristics, and (c) observers remember individual faces to be more alike the ensemble representation than they were, due to hierarchical structure of visual working memory. Across three experiments, we investigated whether the cheerleader effect is consistent with hierarchical encoding, by asking observers to give attractiveness ratings to the same target faces shown in groups and alone. Consistent with hierarchical encoding, the largest attractiveness increases of 1.5-2.0% occurred when target faces were presented in groups of faces that could be mentally summarised to create an ensemble representation with average facial characteristics. Surprisingly, smaller cheerleader effects still occurred in conditions that were incompatible with hierarchical encoding (i.e., groups with non-human images). Together, these results offer only limited evidence for the role of hierarchical encoding in the cheerleader effect, suggesting that alternative mechanisms must be explored in future research.

Visuospatial asymmetries do not modulate the cheerleader effect.

The cheerleader effect occurs when the same individual appears to be more attractive when seen in a group, compared to alone. As observers over-attend to visual information presented in the left visual field, we investigated whether the spatial arrangement of the faces in a group would influence the magnitude of the cheerleader effect. In Experiment 1, target faces were presented twice in the centre of the display: once alone, and once in a group. Group images featured two distractor faces, which were presented in either the left or the right visual field, or on either side of the target. The location of the distractor faces did not modulate the size of the cheerleader effect, which was observed in each group configuration. In Experiment 2, we manipulated the location of the target faces, which were presented at the far left, far right, or centre of the group. Faces were again significantly more attractive in each group configuration, and the spatial location of the target face did not influence the size of the cheerleader effect. Together, our results show that the cheerleader effect is a robust phenomenon, which is not influenced by the spatial arrangement of the faces in the group.

Is trustworthiness lateralized in the face? Evidence from a trust game.

A turn of the head can be used to convey or conceal emotion, as the left side of the face is more expressive than the right. As the left cheek moves more when smiling, the present study investigated whether perceived trustworthiness is lateralized to the left cheek, using a trust game paradigm. In Experiment 1, participants were asked to share money with male and female "virtual partners." Left-left or right-right composite faces were used to represent the partners. There were no differences in the amount shared based on composite face, suggesting trustworthiness is not lateralized in the face. However, there was a robust effect whereby female partners were perceived to be significantly more trustworthy than males. In Experiment 2, the virtual partners presented either the left or the right cheek prominently. As in Experiment 1, the amount shared with the partners did not change depending on the cheek presented. Interestingly, female partners were again sent significantly more money than males. We found no support for lateralized trustworthiness in the face, suggesting that asymmetries in the face are not large enough to influence trustworthiness judgements. Instead, more stable facial features, such as sex-typical characteristics, appear to influence perceived trustworthiness.