Comparing human evaluations of eyewitness statements to a machine learning classifier under pristine and suboptimal lineup administration procedures.

Recent work highlights the ability of verbal machine learning classifiers to distinguish between accurate and inaccurate recognition memory decisions (Dobbins, 2022; Dobbins & Kantner, 2019; Seale-Carlisle, Grabman, & Dodson, 2022). Given the surge of interest in these modeling techniques, there is an urgent need to investigate verbal classifiers' limitations - particularly in applied contexts such as when police collect eyewitness's confidence statements. We find that confirmatory feedback (e.g., "This study now has a total of 87 participants, 84 of them made the same decision as you!") weakens the relationship between identification accuracy and verbal classifier scores to a similar degree as mock witnesses' numeric confidence judgments (Experiment 1). Crucially, for the first time, we compare the discriminative value of verbal classifier scores to the ratings of human evaluators who assessed the identical verbal confidence statements (Experiment 2). Our results suggest that human evaluators outperform the classifier when mock witnesses received no feedback; however, the classifier matches (or exceeds) the performance of human evaluators when mock witnesses received confirmatory feedback. Providing lineup information to human evaluators resulted in a worse ability to distinguish between correct and filler identifications, suggesting that this particular information may encourage the use of inappropriate heuristics when rendering accuracy judgments. Overall, these results suggest that the utility of verbal classifiers may be enhanced when contextual effects (e.g., lineup presence) impair human estimates of others' performance, but that translating witnesses' statements into classifier scores will not fix the problems of an improperly conducted lineup procedure.

New Insights on Expert Opinion About Eyewitness Memory Research.

Experimental psychologists investigating eyewitness memory have periodically gathered their thoughts on a variety of eyewitness memory phenomena. Courts and other stakeholders of eyewitness research rely on the expert opinions reflected in these surveys to make informed decisions. However, the last survey of this sort was published more than 20 years ago, and the science of eyewitness memory has developed since that time. Stakeholders need a current database of expert opinions to make informed decisions. In this article, we provide that update. We surveyed 76 scientists for their opinions on eyewitness memory phenomena. We compared these current expert opinions to expert opinions from the past several decades. We found that experts today share many of the same opinions as experts in the past and have more nuanced thoughts about two issues. Experts in the past endorsed the idea that confidence is weakly related to accuracy, but experts today acknowledge the potential diagnostic value of initial confidence collected from a properly administered lineup. In addition, experts in the past may have favored sequential over simultaneous lineup presentation, but experts today are divided on this issue. We believe this new survey will prove useful to the court and to other stakeholders of eyewitness research.

Unskilled, underperforming, or unaware? Testing three accounts of individual differences in metacognitive monitoring.

Many studies show that competence (e.g., skill, expertise, natural ability) influences individuals' capabilities of monitoring their item-level performance. However, debate persists about how best to explain these individual differences in metacognition. The competence-based account ascribes differences in monitoring to individuals' objective ability level, arguing that the same skills necessary to perform a task are required to effectively monitor performance. The performance-based account attributes differences in monitoring to changes in overall task performance - no individual differences in competence required. Finally, the metacognitive awareness account proposes that alignment between an individuals' self-assessed and objective ability leads to differences in monitoring. In this study, 603 participants completed a self-assessment of face recognition ability, a lineup identification task, and an objective assessment of face recognition ability. We manipulated the number of encoding repetitions and delay between encoding and test to produce varying levels of task performance across objective face recognition ability. Following each lineup decision, participants provided both a numeric confidence rating and a written expression of verbal confidence. We transformed verbal confidence into a quantitative value using machine learning techniques. When matched on overall identification accuracy, objectively stronger face recognizers used numeric and verbal confidence that a) better discriminates between correct and filler lineup identifications than weaker recognizers, and b) shows better calibration to accuracy. Participants with greater self-assessed ability used higher levels of confidence, irrespective of trial accuracy. These results support the competence-based account.

Eyewitness confidence and mock juror decisions of guilt: A meta-analytic review.

OBJECTIVE: We investigated the impact of eyewitness confidence on the following dependent variables: (a) guilty or not-guilty verdict; (b) judgments of guilt as measured on a scale; and (c) mock jurors' perception of the accuracy of an eyewitness's identification. In addition, we examined two potential moderators of the effects of eyewitness confidence: (a) whether the eyewitness expressed confidence at trial versus during the initial lineup identification and (b) whether the eyewitness provided a numerical versus a verbal statement of confidence. HYPOTHESES: We expected all analyses to reveal that highly confident eyewitnesses are more persuasive to mock jurors than are eyewitnesses with lower confidence (Hypothesis 1). We expected eyewitness confidence at trial (relative to at identification) to be more persuasive to mock jurors (Hypothesis 2). We expected numerical expressions of confidence to be more persuasive to mock jurors than verbal confidence expressions (Hypothesis 3). METHOD: We conducted a meta-analysis of 35 studies from 20 published papers and seven theses or dissertations to quantify the effect of eyewitness confidence on juror judgments and investigated the influence of two primary moderator variables, time of confidence and format of confidence expression. RESULTS: All analyses revealed an effect of eyewitness confidence on mock juror decisions (gs = .21-.36). Our moderator analysis showed that the timing of the confidence statement (identification vs. trial) did not affect the influence of eyewitness confidence on mock jurors' judgments of guilt or accuracy. The influence of eyewitness confidence was not moderated by verbal versus numerical expressions of confidence. CONCLUSIONS: Although eyewitness confidence is persuasive to mock jurors, the size of this effect is modest. Moreover, verbal and numerical expressions of confidence have similar persuasive effects, and mock jurors do not appear to be sensitive to the likely difference in evidentiary strength of eyewitness confidence expressed at the initial identification versus at trial. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The language of accurate and inaccurate eyewitnesses.

This article uses machine-learning techniques to examine people's use of verbal expressions of confidence. Across the field of academic psychology, it is often assumed that such statements reflect the same underlying information as numeric confidence ratings. We show that verbal confidence is not redundant with numeric confidence but instead contributes unique diagnostic value in predicting the accuracy of a response. We use eyewitness confidence in a lineup identification as our model paradigm. There is potentially great applied value in developing a machine-learning algorithm that can predict eyewitness identification accuracy, such as by reducing false convictions. To this end, we applied a machine-learning methodology to investigate the natural language of accurate and inaccurate eyewitnesses. This method revealed that verbal confidence statements provide rich diagnostic information about the likely accuracy of eyewitness identifications. Moreover, verbal confidence statements provide unique diagnostic information that traditional indicators of identification accuracy such as numeric confidence ratings and response times do not provide. However, the diagnostic value of an eyewitness confidence statement depends in part on the face recognition ability of the eyewitness: The natural language of strong face recognizers is more diagnostic than the natural language of weak face recognizers. These results are theoretically interesting but, from an applied perspective, this machine-learning methodology may prove useful to those in the criminal justice system who must evaluate eyewitnesses' verbal confidence statements. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Improving the interpretation of verbal eyewitness confidence statements by distinguishing perceptions of certainty from those of accuracy.

When an eyewitness makes an identification from a lineup, police are also instructed to collect a verbal expression of confidence. This recommendation hinges on the assumption that evaluators will perceive confidence in the manner the witness intended. However, research has consistently shown that these interpretations can be biased by accompanying contextual information. For example, statements that reference facial features (e.g., "I'm very sure. I remember his eyes.") are perceived as less confident than when the statement is presented alone ("I'm very sure.") (featural justification effect). Additionally, perceptions of witness confidence are altered when the witness's identification (mis-)matches the police suspect in a lineup (prior knowledge). We find that the same underlying mechanism explains the bias induced by both featural justification (Experiments 1 and 2) and prior knowledge (Experiment 3) manipulations. Evaluators conflate their own beliefs about the accuracy of an identification with the witness's intended level of confidence. A simple warning that highlights the differences between confidence and accuracy eliminates the featural justification effect, but is less effective for mitigating the influence of prior knowledge. The key takeaway from this paper is that distinguishing perceptions of certainty from those of accuracy improves the interpretation of verbal confidence statements. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

A decision processes account of the differences in the eyewitness confidence-accuracy relationship between strong and weak face recognizers under suboptimal exposure and delay conditions.

When pristine testing conditions are used, an eyewitness's high-confidence identification from a lineup can be a reliable predictor of their identification accuracy (Wixted & Wells, 2017). Further, Grabman, Dobolyi, Berelovich, and Dodson (2019) found that high-confidence identifications are more predictive of accuracy for individuals with stronger than weaker face recognition ability. We extend this research by investigating why strong face recognizers make more informative confidence judgments and fewer high-confidence errors through the framework of two different accounts: the optimality account (Deffenbacher, 1980) and the decision processes account (e.g., Kruger & Dunning, 1999). The optimality account holds that differences in the predictive value of confidence ratings made by strong versus weak face recognizers are a result of differences in the quality of their memory representations for faces, indicating that confidence-accuracy calibration would be equated between these two groups when overall accuracy is equated, whereas the decision processes account attributes differences in calibration to strong face recognizers' superior metacognitive skills, which allow them to better evaluate their performance in the domain of face recognition. Therefore, to distinguish between these accounts, we manipulated exposure and retention interval to create conditions that produced comparable levels of identification accuracy between stronger and weaker face recognizers, and then examined their confidence-accuracy calibration. The decision processes account was supported, as differences in calibration between stronger and weaker face recognizers persisted even when overall identification accuracy was equated. Stronger face recognizers are better able to regulate their use of the confidence scale points with changes in identification accuracy. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

SUAG/on across the adult lifespan.

One of the central concepts within the literature on cognitive aging is the notion of dedifferentiation-the idea that increasing age is associated with an increase in the interrelatedness of different cognitive abilities. Despite the centrality of this dedifferentiation hypothesis, there is a great deal of evidence that both supports and does not support dedifferentiation. We hypothesized that these inconsistent findings were due to (a) the use of different cognitive abilities (i.e., memory vs. speed of processing) that were correlated; and (b) the differing age groups that were used across studies. By using data from 11 well-validated cognitive test batteries (K = 2,355, range of the mean ages of correlations 18-85+), we found evidence for linear dedifferentiation when a test assessing speed of processing was included in the correlation with test of other cognitive abilities. We speculate that previous findings of nonlinear dedifferentiation are likely a result of undiagnosed or unrecognized pathology in a subsample of participants. (PsycINFO Database Record

Actual vs. perceived eyewitness accuracy and confidence and the featural justification effect.

This article documents a contradiction between objective eyewitness accuracy and perceived eyewitness accuracy. Objectively, eyewitness identification accuracy (and the confidence-accuracy relationship) is comparably strong when a lineup identification is accompanied by a justification that refers to either an observable feature about the suspect ("I remember his eyes"), an unobservable feature ("He looks like a friend of mine") or just a statement of recognition ("I recognize him"). There is, however, a weaker relationship between confidence and accuracy and an increase in high confidence errors for identifications that are accompanied by references to familiarity than by references to any other type of justification. With respect to perceived accuracy, we document a robust cognitive bias-the featural justification effect-that causes eyewitnesses to be regarded by others as less accurate and less confident when they justify their identification by referring to an observable feature as compared to when they give any other kind of justification, except for a reference to familiarity. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Recollection is a continuous process: Evidence from plurality memory receiver operating characteristics.

Is recollection a continuous/graded process or a threshold/all-or-none process? Receiver operating characteristic (ROC) analysis can answer this question as the continuous model and the threshold model predict curved and linear recollection ROCs, respectively. As memory for plurality, an item's previous singular or plural form, is assumed to rely on recollection, the nature of recollection can be investigated by evaluating plurality memory ROCs. The present study consisted of four experiments. During encoding, words (singular or plural) or objects (single/singular or duplicate/plural) were presented. During retrieval, old items with the same plurality or different plurality were presented. For each item, participants made a confidence rating ranging from "very sure old", which was correct for same plurality items, to "very sure new", which was correct for different plurality items. Each plurality memory ROC was the proportion of same versus different plurality items classified as "old" (i.e., hits versus false alarms). Chi-squared analysis revealed that all of the plurality memory ROCs were adequately fit by the continuous unequal variance model, whereas none of the ROCs were adequately fit by the two-high threshold model. These plurality memory ROC results indicate recollection is a continuous process, which complements previous source memory and associative memory ROC findings.

Misinterpreting eyewitness expressions of confidence: The featural justification effect.

How do we know eyewitness statements of confidence are interpreted accurately by others? When eyewitnesses provide a verbal expression of confidence about a lineup identification, such as I'm fairly certain it's him, how well do others understand the intended meaning of this statement of confidence? And, how is this perception of the meaning influenced by justifications of the level of confidence, such as when eyewitnesses say, I remember his chin? The answers to these questions are unknown, as there is no research on how others interpret the intended meaning of eyewitness confidence. Three experiments show that an additional justification of confidence, relative to seeing a confidence statement alone, can increase misunderstanding in others' estimation of the meaning of the expression of confidence. Moreover, this justification-induced increase in misunderstanding only occurs when the justification refers to an observable facial feature and not when it refers to an unobservable quality (e.g., He is very familiar). Even more noteworthy, both Experiments 2 and 3 show that this featural justification effect is strongest when eyewitnesses express absolute certainty in an identification, such as by stating I am positive. When a highly confident assertion is accompanied by a featural justification others will be most likely to misinterpret the intended meaning.

Aging, confidence, and misinformation: recalling information with the cognitive interview.

In 2 experiments, younger and older adults witnessed a simulated robbery, received misleading information about the event, and then were interviewed with the Cognitive Interview about their memory for the robbery. In both experiments, older adults were disproportionately more confident than younger adults in the accuracy of incorrect information that they recalled than in the accuracy of correct information. Critically, this age-related increase in high-confidence errors occurred even in comparison with younger adults who were matched with older adults on the overall amount and accuracy of the information remembered about the robbery. In addition, Experiment 2 showed that retrieval warnings to disregard the misinformation were just as effective in older adults as compared with younger adults at reducing the reporting of misleading information. Finally, both experiments showed that across the multiple retrieval stages of the Cognitive Interview, the final retrieval stage is roughly half as effective for older adults relative to younger adults at eliciting previously unreported information. These results indicate that investigators have much less to gain from older adults than they do from younger adults with repeated inquiries (during the same session) about a witnessed event.

Eyewitness confidence in simultaneous and sequential lineups: a criterion shift account for sequential mistaken identification overconfidence.

Confidence judgments for eyewitness identifications play an integral role in determining guilt during legal proceedings. Past research has shown that confidence in positive identifications is strongly associated with accuracy. Using a standard lineup recognition paradigm, we investigated accuracy using signal detection and ROC analyses, along with the tendency to choose a face with both simultaneous and sequential lineups. We replicated past findings of reduced rates of choosing with sequential as compared to simultaneous lineups, but notably found an accuracy advantage in favor of simultaneous lineups. Moreover, our analysis of the confidence-accuracy relationship revealed two key findings. First, we observed a sequential mistaken identification overconfidence effect: despite an overall reduction in false alarms, confidence for false alarms that did occur was higher with sequential lineups than with simultaneous lineups, with no differences in confidence for correct identifications. This sequential mistaken identification overconfidence effect is an expected byproduct of the use of a more conservative identification criterion with sequential than with simultaneous lineups. Second, we found a steady drop in confidence for mistaken identifications (i.e., foil identifications and false alarms) from the first to the last face in sequential lineups, whereas confidence in and accuracy of correct identifications remained relatively stable. Overall, we observed that sequential lineups are both less accurate and produce higher confidence false identifications than do simultaneous lineups. Given the increasing prominence of sequential lineups in our legal system, our data argue for increased scrutiny and possibly a wholesale reevaluation of this lineup format.

Cross-age effect in recognition performance and memory monitoring for faces.

The cross-age effect refers to the finding of better memory for own- than other-age faces. We examined 3 issues about this effect: (1) Does it extend to the ability to monitor the likely accuracy of memory judgments for young and old faces? (2) Does it apply to source information that is associated with young and old faces? And (3) what is a likely mechanism underlying the cross-age effect? In Experiment 1, young and older adults viewed young and old faces appearing in different contexts. Young adults exhibited a cross-age effect in their recognition of faces and in their memory-monitoring performance for these faces. Older adults, by contrast, showed no age-of-face effects. Experiment 2 examined whether young adults' cross-age effect depends on or is independent of encoding a mixture of young and old faces. Young adults encoded either a mixture of young and old faces, a set of all young faces, or a set of all old faces. In the mixed-list condition we replicated our finding of young adults' superior memory for own-age faces; in the pure-list conditions, however, there were absolutely no differences in performance between young and old faces. The fact that the pure-list design abolishes the cross-age effect supports social-cognitive theories of this phenomenon.

Negative prospective memory: remembering not to perform an action.

Sometimes it is important to remember not to perform an action, such as remembering to stop taking seasonal allergy medicine when it is no longer needed. Mistakes in accomplishing this goal can involve prospective memory commission errors when individuals mistakenly perform a prospective response. In two experiments, we investigated the role of attentional resources in preventing prospective memory errors to cues that had been associated with a habitual prospective response. In Phase 1 of our experiments, participants performed a prospective memory task during which they either routinely responded to prospective memory cues or responded to these cues one time only. On a subsequent prospective memory task, the participants who had routinely responded to prospective memory cues were vulnerable to commission errors when their attentional resources were taxed. By contrast, dividing attention did not increase commission errors to cues that had not been routinely performed on an earlier task. These data indicate that attentional resources are needed to withhold making habitual prospective memory responses, and they suggest that inhibitory failures are a cause of these prospective memory errors. Finally, we suggest a broader definition of prospective memory that includes remembering to withhold performing actions.

Alzheimer's disease and memory-monitoring impairment: Alzheimer's patients show a monitoring deficit that is greater than their accuracy deficit.

We assessed the ability of two groups of patients with mild Alzheimer's disease (AD) and two groups of older adults to monitor the likely accuracy of recognition judgments and source identification judgments about who spoke something earlier. Alzheimer's patients showed worse performance on both memory judgments and were less able to monitor with confidence ratings the likely accuracy of both kinds of memory judgments, as compared to a group of older adults who experienced the identical study and test conditions. Critically, however, when memory performance was made comparable between the AD patients and the older adults (e.g., by giving AD patients extra exposures to the study materials), AD patients were still greatly impaired at monitoring the likely accuracy of their recognition and source judgments. This result indicates that the monitoring impairment in AD patients is actually worse than their memory impairment, as otherwise there would have been no differences between the two groups in monitoring performance when there were no differences in accuracy. We discuss the brain correlates of this memory-monitoring deficit and also propose a Remembrance-Evaluation model of memory-monitoring.

Metamemory development: understanding the role of similarity in false memories.

Research on the development of metamemory has focused primarily on children's understanding of the variables that influence how likely a person is to remember something. But metamemory also involves an understanding of why people occasionally misremember things. In this study, 5- and 6-year-olds (N = 38) were asked to decide whether another child's mistakes in a memory game were due to false memories or guesses. Some of the fictitious child's mistakes were similar to material he had seen earlier and some were not. Six-year-olds, but not 5-year-olds, consistently attributed more similar than dissimilar mistakes to false memories. Understanding the link between similarity and false memories improves significantly between 5 and 6 years of age.

Stereotypes and retrieval-provoked illusory source recollections.

When expectations and stereotypes are activated at retrieval, they spontaneously create distorted and illusory recollections that are consistent with these expectations. Participants studied doctor (physician)-related and lawyer-related statements that were presented by 2 different people. When informed, on a subsequent source memory test, (i.e., of who presented what) that one of the study sources was actually a doctor and the other source was a lawyer, there was a strong tendency to attribute the test items in a stereotype-consistent manner. In 3 experiments, participants frequently reported recollecting specific details, such as via "remember" judgments, to justify their stereotype-consistent but incorrect responses. These experiments rule out explanations involving either the misattribution of strong familiarity or differences in the bias to making remember responses as accounts for the illusory source attributions. Instead, the illusory recollections are consistent with the notion that recollective experience is manufactured from both the information in the memory trace and information in the retrieval environment, such as an individual's expectations, stereotypes, and general knowledge.

Retrieval-based illusory recollections: why study-test contextual changes impair source memory.

Illusory source recollections can be manufactured purely by the conditions at retrieval. At encoding, individuals listened to words spoken by a male or a female voice and later, at retrieval, determined who initially had spoken the test word or whether it was a new word. In Experiment 1, individuals were instructed during the memory test to make a response of male or female only if they clearly recollected the speaker presenting the item during encoding and to respond don't remember otherwise. Presenting test words with the same voice (match condition) as the one that had presented the word at encoding, a different voice (mismatch condition), or no test voice had no effect on the rate of responding don't remember. Instead, matching and mismatching test voices created illusory recollections that were consistent with the test voice. Experiment 2 yielded similar results with a remember/know source test. In this article, a theory is proposed that explains the illusory recollection effects, and a multinomial model and procedure are used to separate and measure the contributions of source discrimination and illusory recollections to performance.

When two heads are not better than one: partner neglect in paired memory tasks.

People can monitor the accuracy of their own memories and regulate their responses accordingly. But can they monitor and make use of another person's memory? We document a new phenomenon whereby participants neglect a partner's expertise when deciding whether to defer to that partner's memory or to rely on their own. In two experiments, participants studied images for more time than, less time than, or the same amount of time as a partner, and on subsequent recognition tests they were directed to maximize team performance by either answering themselves or letting their partner respond. In both experiments, individuals failed to use the knowledge that the partner would probably have a better memory for certain items. Only when explicitly instructed to estimate their accuracy relative to their partner's did participants take advantage of the partner's greater expertise.