Understanding 'error' in the forensic sciences: A primer.

This paper distils seven key lessons about 'error' from a collaborative webinar series between practitioners at Victoria Police Forensic Services Department and academics. It aims to provide the common understanding of error necessary to foster interdisciplinary dialogue, collaboration and research. The lessons underscore the inevitability, complexity and subjectivity of error, as well as opportunities for learning and growth. Ultimately, we argue that error can be a potent tool for continuous improvement and accountability, enhancing the reliability of forensic sciences and public trust. It is hoped the shared understanding provided by this paper will support future initiatives and funding for collaborative developments in this vital domain.

Modelling the impact of single vs. dual presentation on visual discrimination across resolutions.

Visual categorisation relies on our ability to extract useful diagnostic information from complex stimuli. To do this, we can utilise both the "high-level" and "low-level" information in a stimulus; however, the extent to which changes in these properties impact the decision-making process is less clear. We manipulated participants' access to high-level category features via gradated reductions to image resolution while exploring the impact of access to additional category features through a dual-stimulus presentation when compared with single stimulus presentation. Results showed that while increasing image resolution consistently resulted in better choice performance, no benefit was found for dual presentation over single presentation, despite responses for dual presentation being slower compared with single presentation. Applying the diffusion decision model revealed increases in drift rate as a function of resolution, but no change in drift rate for single versus dual presentation. The increase in response time for dual presentation was instead accounted for by an increase in response caution for dual presentations. These findings suggest that while increasing access to high-level features (via increased resolution) can improve participants' categorisation performance, increasing access to both high- and low-level features (via an additional stimulus) does not.

A guide to measuring expert performance in forensic pattern matching.

Decisions in forensic science are often binary. A firearms expert must decide whether a bullet was fired from a particular gun or not. A face comparison expert must decide whether a photograph matches a suspect or not. A fingerprint examiner must decide whether a crime scene fingerprint belongs to a suspect or not. Researchers who study these decisions have therefore quantified expert performance using measurement models derived largely from signal detection theory. Here we demonstrate that the design and measurement choices researchers make can have a dramatic effect on the conclusions drawn about the performance of forensic examiners. We introduce several performance models - proportion correct, diagnosticity ratio, and parametric and non-parametric signal detection measures - and apply them to forensic decisions. We use data from expert and novice fingerprint comparison decisions along with a resampling method to demonstrate how experimental results can change as a function of the task, case materials, and measurement model chosen. We also graphically show how response bias, prevalence, inconclusive responses, floor and ceiling effects, case sampling, and number of trials might affect one's interpretation of expert performance in forensics. Finally, we discuss several considerations for experimental and diagnostic accuracy studies: (1) include an equal number of same-source and different-source trials; (2) record inconclusive responses separately from forced choices; (3) include a control comparison group; (4) counterbalance or randomly sample trials for each participant; and (5) present as many trials to participants as is practical.

The effect of fingerprint expertise on visual short-term memory.

Expert fingerprint examiners demonstrate impressive feats of memory that may support their accuracy when making high-stakes identification decisions. Understanding the interplay between expertise and memory is therefore critical. Across two experiments, we tested fingerprint examiners and novices on their visual short-term memory for fingerprints. In Experiment 1, experts showed substantially higher memory performance compared to novices for fingerprints from their domain of expertise. In Experiment 2, we manipulated print distinctiveness and found that while both groups benefited from distinctive prints, experts still outperformed novices. This indicates that beyond stimulus qualities, expertise itself enhances short-term memory, likely through more effective organisational processing and sensitivity to meaningful patterns. Taken together, these findings shed light on the cognitive mechanisms that may explain fingerprint examiners' superior memory performance within their domain of expertise. They further suggest that training to improve memory for diverse fingerprints could practically boost examiner performance. Given the high-stakes nature of forensic identification, characterising psychological processes like memory that potentially contribute to examiner accuracy has important theoretical and practical implications.

Insight and the selection of ideas.

Perhaps it is no accident that insight moments accompany some of humanity's most important discoveries in science, medicine, and art. Here we propose that feelings of insight play a central role in (heuristically) selecting an idea from the stream of consciousness by capturing attention and eliciting a sense of intuitive confidence permitting fast action under uncertainty. The mechanisms underlying this Eureka heuristic are explained within an active inference framework. First, implicit restructuring via Bayesian reduction leads to a higher-order prediction error (i.e., the content of insight). Second, dopaminergic precision-weighting of the prediction error accounts for the intuitive confidence, pleasure, and attentional capture (i.e., the feeling of insight). This insight as precision account is consistent with the phenomenology, accuracy, and neural unfolding of insight, as well as its effects on belief and decision-making. We conclude by reflecting on dangers of the Eureka Heuristic, including the arising and entrenchment of false beliefs and the vulnerability of insights under psychoactive substances and misinformation.

The invisible 800-pound gorilla: expertise can increase inattentional blindness.

People can fail to notice objects and events in their visual environment when their attention is engaged elsewhere. This phenomenon is known as inattentional blindness, and its consequences can be costly for important real-world decisions. However, not noticing certain visual information could also signal expertise in a domain. In this study, we compared professional fingerprint analysts and novices on a fingerprint matching task in which we covertly placed an image of a gorilla into one of the prints. This gorilla was either small, or large, but always embedded in a way that made it largely irrelevant to the primary task. We found that analysts were more likely than the novices to miss the large gorilla. We interpret this finding not as a flaw in how these experts make decisions, but most likely an expression of their expertise; instead of processing more information they filter out irrelevant information and constrain their attention to what is important.

The illusion of insight: detailed warnings reduce but do not prevent false "Aha!" moments.

False "Aha!" moments can be elicited experimentally using the False Insight Anagram Task (FIAT), which combines semantic priming and visual similarity manipulations to lead participants into having "Aha!" moments for incorrect anagram solutions. In a preregistered experiment (N = 255), we tested whether warning participants and explaining to them exactly how they were being deceived, would reduce their susceptibility to false insights. We found that simple warnings did not reduce the incidence of false insights. On the other hand, participants who were given a detailed explanation of the methods used to deceive them experienced a small reduction in false insights compared to participants given no warning at all. Our findings suggest that the FIAT elicits a robust false insight effect that is hard to overcome, demonstrating the persuasive nature of false insights when the conditions are ripe for them.

Specific versus varied practice in perceptual expertise training.

We used a longitudinal randomized control experiment to compare the effect of specific practice (training on one form of a task) and varied practice (training on various forms of a task) on perceptual learning and transfer. Participants practiced a visual search task for 10 hours over 2 to 4 weeks. The specific practice group searched for features only in fingerprints during each session, whereas the varied practice group searched for features in five different image categories. Both groups were tested on a series of tasks at four time points: before training, midway through training, immediately after training ended, and 6 to 8 weeks later. The specific group improved more during training and demonstrated greater pre-post performance gains than the varied group on a visual search task with untrained fingerprint images. Both groups improved equally on a visual search task with an untrained image category, but only the specific group's performance dropped significantly when tested several weeks later. Finally, both groups improved equally on a series of untrained fingerprint tasks. Practice with respect to a single category (versus many) instills better near transfer, but category-specific and category-general visual search training appear equally effective for developing task-general expertise. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Thinking style and psychosis proneness do not predict false insights.

The FIAT paradigm (Grimmer et al., 2021) is a novel method of eliciting 'Aha' moments for incorrect solutions to anagrams in the laboratory, i.e. false insights. There exist many documented reports of psychotic symptoms accompanying strong feelings of 'Aha!' (Feyaerts, Henriksen, Vanheule, Myin-Germeys, & Sass, 2021; Mishara, 2010; Tulver, Kaup, Laukkonen, & Aru, 2021), suggesting that the newly developed FIAT could reveal whether people who have more false insights are more prone to psychosis and delusional belief. To test this possibility, we recruited 200 participants to take an adapted version of the FIAT and complete measures of thinking style and psychosis proneness. We found no association between experimentally induced false insights and measures of Schizotypy, Need for Cognition, Jumping to Conclusions, Aberrant Salience, Faith in Intuition, or the Cognitive Reflection Task. We conclude that experiencing false insights might not be constrained to any particular type of person, but rather, may arise for anyone under the right circumstances.

Irrelevant insights make worldviews ring true.

Our basic beliefs about reality can be impossible to prove and yet we can feel a strong intuitive conviction about them, as exemplified by insights that imbue an idea with immediate certainty. Here we presented participants with worldview beliefs such as "people's core qualities are fixed" and simultaneously elicited an aha moment. In the first experiment (N = 3000, which included a direct replication), participants rated worldview beliefs as truer when they solved anagrams and also experienced aha moments. A second experiment (N = 1564) showed that the worldview statement and the aha moment must be perceived simultaneously for this 'insight misattribution' effect to occur. These results demonstrate that artificially induced aha moments can make worldview beliefs seem truer, possibly because humans partially rely on feelings of insight to appraise an idea's veracity. Feelings of insight are therefore not epiphenomenal and should be investigated for their effects on decisions, beliefs, and delusions.

Getting a grip on insight: real-time and embodied Aha experiences predict correct solutions.

Insight experiences are sudden, persuasive, and can accompany valuable new ideas in science and art. In this preregistered experiment, we aim to validate a novel visceral and continuous measure of insight problem solving and to test whether real-time and embodied feelings of insight can predict correct solutions. We report several findings. Consistent with recent work, we find a strong positive relationship between Aha moments and accuracy for problems that demand implicit processing. We also found that the intensity of the insight experience further predicted the accuracy of solutions and participants naturally embodied the intensity of their insight experiences by squeezing the dynamometer more tightly. Intriguingly, this unintentional embodiment further predicted the accuracy of solutions. We suggest that the dynamometer complements previous measures by (1) simultaneously capturing both process and feeling in real-time, (2) highlights the value of measuring Aha moments on a continuum of intensity, and (3) firmly establishes that the impulsive feeling of Aha can carry information about the veracity of an idea. We discuss the findings in light of a recent theoretical account of how feelings of insight may act as a heuristic to select ideas from the stream of consciousness.

The effect of expertise, target usefulness and image structure on visual search.

Experts outperform novices on many cognitive and perceptual tasks. Extensive training has tuned experts to the most relevant information in their specific domain, allowing them to make decisions quickly and accurately. We compared a group of fingerprint examiners to a group of novices on their ability to search for information in fingerprints across two experiments-one where participants searched for target features within a single fingerprint and another where they searched for points of difference between two fingerprints. In both experiments, we also varied how useful the target feature was and whether participants searched for these targets in a typical fingerprint or one that had been scrambled. Experts more efficiently located targets when searching for them in intact but not scrambled fingerprints. In Experiment 1, we also found that experts more efficiently located target features classified as more useful compared to novices, but this expert-novice difference was not present when the target feature was classified as less useful. The usefulness of the target may therefore have influenced the search strategies that participants used, and the visual search advantages that experts display appear to depend on their vast experience with visual regularity in fingerprints. These results align with a domain-specific account of expertise and suggest that perceptual training ought to involve learning to attend to task-critical features.

Collective intelligence in fingerprint analysis.

When a fingerprint is located at a crime scene, a human examiner is counted upon to manually compare this print to those stored in a database. Several experiments have now shown that these professional analysts are highly accurate, but not infallible, much like other fields that involve high-stakes decision-making. One method to offset mistakes in these safety-critical domains is to distribute these important decisions to groups of raters who independently assess the same information. This redundancy in the system allows it to continue operating effectively even in the face of rare and random errors. Here, we extend this "wisdom of crowds" approach to fingerprint analysis by comparing the performance of individuals to crowds of professional analysts. We replicate the previous findings that individual experts greatly outperform individual novices, particularly in their false-positive rate, but they do make mistakes. When we pool the decisions of small groups of experts by selecting the decision of the majority, however, their false-positive rate decreases by up to 8% and their false-negative rate decreases by up to 12%. Pooling the decisions of novices results in a similar drop in false negatives, but increases their false-positive rate by up to 11%. Aggregating people's judgements by selecting the majority decision performs better than selecting the decision of the most confident or the most experienced rater. Our results show that combining independent judgements from small groups of fingerprint analysts can improve their performance and prevent these mistakes from entering courts.

The dark side of Eureka: Artificially induced Aha moments make facts feel true.

Some ideas that we have feel mundane, but others are imbued with a sense of profundity. We propose that Aha! moments make an idea feel more true or valuable in order to aid quick and efficient decision-making, akin to a heuristic. To demonstrate where the heuristic may incur errors, we hypothesized that facts would appear more true if they were artificially accompanied by an Aha! moment elicited using an anagram task. In a preregistered experiment, we found that participants (n=300) provided higher truth ratings for statements accompanied by solved anagrams even if the facts were false, and the effect was particularly pronounced when participants reported an Aha! experience (d = .629). Recent work suggests that feelings of insight usually accompany correct ideas. However, here we show that feelings of insight can be overgeneralized and bias how true an idea or fact appears, simply if it occurs in the temporal 'neighbourhood' of an Aha! moment. We raise the possibility that feelings of insight, epiphanies, and Aha! moments have a dark side, and discuss some circumstances where they may even inspire false beliefs and delusions, with potential clinical importance.

How low can you go? Detecting style in extremely low resolution images.

Humans can see through the complexity of scenes, faces, and objects by quickly extracting their redundant low-spatial and low-dimensional global properties, or their style. It remains unclear, however, whether semantic coding is necessary, or whether visual stylistic information is sufficient, for people to recognize and discriminate complex images and categories. In two experiments, we systematically reduce the resolution of hundreds of unique paintings, birds, and faces, and test people's ability to discriminate and recognize them. We show that the stylistic information retained at extremely low image resolutions is sufficient for visual recognition of images and visual discrimination of categories. Averaging over the 3 domains, people were able to reliably recognize images reduced down to a single pixel, with large differences from chance discriminability across 8 different image resolutions. People were also able to discriminate categories substantially above chance with an image resolution as low as 2 × 2 pixels. We situate our findings in the context of contemporary computational accounts of visual recognition and contend that explicit encoding of the local features in the image, or knowledge of the semantic category, is not necessary for recognizing and distinguishing complex visual stimuli. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Beliefs about error rates and human judgment in forensic science.

Forensic science techniques are often used in criminal trials to infer the identity of the perpetrator of crime and jurors often find this evidence very persuasive. Unfortunately, two of the leading causes of wrongful convictions are forensic science testing errors and false or misleading forensic testimony (Saks and Koehler, 2005). Therefore, it is important to understand jurors' pre-existing beliefs about forensic science, as these beliefs may impact how they evaluate forensic evidence in the courtroom. In this study, we examine people's perceptions of the likelihood of error and human judgment involved at each stage of the forensic science process (i.e., collection, storage, testing, analysis, reporting, and presenting). In addition, we examine people's perceptions of the accuracy of - and human judgment involved in - 16 different forensic techniques. We find that, in contrast to what would be expected by the CSI effect literature, participants believed that the process of forensic science involved considerable human judgment and was relatively error-prone. In addition, participants had wide-ranging beliefs about the accuracy of various forensic techniques, ranging from 65.18% (document analysis) up to 89.95% (DNA). For some forensic techniques, estimates were lower than that found in experimental proficiency studies, suggesting that our participants are more skeptical of certain forensic evidence than they need to be.

How to Detect Insight Moments in Problem Solving Experiments.

Arguably, it is not possible to study insight moments during problem solving without being able to accurately detect when they occur (Bowden and Jung-Beeman, 2007). Despite over a century of research on the insight moment, there is surprisingly little consensus on the best way to measure them in real-time experiments. There have also been no attempts to evaluate whether the different ways of measuring insight converge. Indeed, if it turns out that the popular measures of insight diverge, then this may indicate that researchers who have used one method may have been measuring a different phenomenon to those who have used another method. We compare the strengths and weaknesses of the two most commonly cited ways of measuring insight: The feelings-of-warmth measure adapted from Metcalfe and Wiebe (1987), and the self-report measure adapted from Bowden and Jung-Beeman (2007). We find little empirical agreement between the two measures, and conclude that the self-report measure of Aha! is superior both methodologically and theoretically, and provides a better representation of what is commonly regarded as insight. We go on to describe and recommend a novel visceral measure of insight using a dynamometer as described in Creswell et al. (2016).

A visual familiarity account of evidence for orthographic processing in pigeons (Columbia livia): a reply to Scarf, Corballis, Güntürkün, and Colombo (2017).

Scarf et al. (Proc Natl Acad Sci 113(40):11272-11276, 2016) demonstrated that pigeons, as with baboons (Grainger et al. in Science 336(6078):245-248, 2012; Ziegler in Psychol Sci. https://doi.org/10.1177/0956797612474322 , 2013), can be trained to display several behavioural hallmarks of human orthographic processing. But, Vokey and Jamieson (Psychol Sci 25(4):991-996, 2014) demonstrated that a standard, autoassociative neural network model of memory applied to pixel maps of the words and nonwords reproduces all of those results. In a subsequent report, Scarf et al. (Anim Cognit 20(5):999-1002, 2017) demonstrated that pigeons can reproduce one more marker of human orthographic processing: the ability to discriminate visually presented four-letter words from their mirror-reversed counterparts (e.g. "LEFT" vs. " "). The current report shows that the model of Vokey and Jamieson (2014) reproduces the results of Scarf et al. (2017) and reinforces the original argument: the recent results thought to support a conclusion of orthographic processing in pigeons and baboons are consistent with but do not force that conclusion.

Expertise with unfamiliar objects is flexible to changes in task but not changes in class.

Perceptual expertise is notoriously specific and bound by familiarity; generalizing to novel or unfamiliar images, objects, identities, and categories often comes at some cost to performance. In forensic and security settings, however, examiners are faced with the task of discriminating unfamiliar images of unfamiliar objects within their general domain of expertise (e.g., fingerprints, faces, or firearms). The job of a fingerprint expert, for instance, is to decide whether two unfamiliar fingerprint images were left by the same unfamiliar finger (e.g., Smith's left thumb), or two different unfamiliar fingers (e.g., Smith and Jones's left thumb). Little is known about the limits of this kind of perceptual expertise. Here, we examine fingerprint experts' and novices' ability to distinguish fingerprints compared to inverted faces in two different tasks. Inverted face images serve as an ideal comparison because they vary naturally between and within identities, as do fingerprints, and people tend to be less accurate or more novice-like at distinguishing faces when they are presented in an inverted or unfamiliar orientation. In Experiment 1, fingerprint experts outperformed novices in locating categorical fingerprint outliers (i.e., a loop pattern in an array of whorls), but not inverted face outliers (i.e., an inverted male face in an array of inverted female faces). In Experiment 2, fingerprint experts were more accurate than novices at discriminating matching and mismatching fingerprints that were presented very briefly, but not so for inverted faces. Our data show that perceptual expertise with fingerprints can be flexible to changing task demands, but there can also be abrupt limits: fingerprint expertise did not generalize to an unfamiliar class of stimuli. We interpret these findings as evidence that perceptual expertise with unfamiliar objects is highly constrained by one's experience.