Polygraph-based deception detection and Machine Learning. Combining the Worst of Both Worlds?

At a time when developments in computational approaches, often associated with the now much-vaunted terms Machine Learning (ML) and Artificial Intelligence (AI), face increasing challenges in terms of fairness, transparency and accountability, the temptation for researchers to apply mainstream ML methods to virtually any type of data seems to remain irresistible. In this paper we critically examine a recent proposal to apply ML to polygraph screening results (where human interviewers have made a conclusion about deception), which raises several questions about the purpose and the design of the research, particularly given the vacuous scientific status of polygraph-based procedures themselves. We argue that in high-stake environments such as criminal justice and employment practice, where fundamental rights and principles of justice are at stake, the legal and ethical considerations for scientific research are heightened. Specifically, we argue that the combination of ambiguously labelled data and ad hoc ML models does not meet this requirement. Worse, such research can inappropriately legitimise otherwise scientifically invalid, indeed pseudo-scientific methods such as polygraph-based deception detection, especially when presented in a reputable scientific journal. We conclude that methodological concerns, such as those highlighted in this paper, should be addressed before research can be said to contribute to resolving any of the fundamental validity issues that underlie methods and techniques used in legal proceedings.

The strange persistence of (source) "identification" claims in forensic literature through descriptivism, diagnosticism and machinism.

Many forensic scientists consider that identification (individualisation) - in the sense of statements of the kind "the questioned item and the known item come from the same source" - is a concept that is central to their discipline. This is so despite decade-long, fundamental critiques levelled by both practitioners and academics against the conceptual and practical feasibility of forensic identification. Oddly, there is a constant stream of publications in (peer-reviewed) forensic science journals that treat forensic identification axiomatically as a valid object of study, sidestepping the fundamental critiques. This paper reviews and discusses three exemplary strands of publications that exemplify this persistent trend. These strands are called descriptivism, diagnosticism and machinism. The latter term refers to methods borrowed from the now increasingly popular approaches used in the field of machine learning. In turn, descriptivism and diagnosticism refer to general design aspects of mainstream research methods, illustrated here through a critical review of two recent papers on, respectively, forensic odontology and a framework for interpreting fingerprint evidence. The critique of the use of 'identification' in these strands of publication includes, but goes beyond, semantic details and the reiteration of long-known shortcomings of obsolete technical language such as 'match' and 'matching'. Specifically, this paper exposes deeper problems such as the subtle and argumentatively unfounded carrying-over of source conclusions to ultimate issues and the use probability concepts for questions that require more than the mere quantification of uncertainty. This paper submits that in order to foster trust in an era of continually expanding publishing activities, it should be a vital interest to forensic science journals to better examine what identification-related research can and cannot legitimately purport to achieve.

Forensic science and the principle of excluded middle: "Inconclusive" decisions and the structure of error rate studies.

In a paper published recently in this journal, Dror and Scurich (2020) [20] critically discuss the notions of "inconclusive evidence" (i.e., test items for which it is difficult to render a categorical response) and "inconclusive decisions" (i.e., experts' conclusions or responses) in the context of forensic science error rate studies. They expose several ways in which the understanding and use of "inconclusives" in current forensic science research and practice can adversely affect the outcomes of error rate studies. A main cause of distortion, according to Dror and Scurich, is what they call "erroneous inconclusive" decisions, in particular the lack of acknowledgment of this type of erroneous conclusion in the computation of error rates. To overcome this complication, Dror and Scurich call for a more explicit monitoring of "inconclusives" using a modified error rate study design. Whilst we agree with several well-argued points raised by the authors, we disagree with their framing of "inconclusive decisions" as potential errors. In this paper, we argue that referring to an "inconclusive decision" as an error is a contradiction in terms, runs counter to an analysis based on decision logic and, hence, is questionable as a concept. We also reiterate that the very term "inconclusive decision" disregards the procedural architecture of the criminal justice system across modern jurisdictions, especially the fact that forensic experts have no decisional rights in the criminal process. These positions do not ignore the possibility that "inconclusives" - if used excessively - do raise problems in forensic expert reporting, in particular limited assertiveness (or, overcautiousness). However, these drawbacks derive from inherent limitations of experts rather than from the seemingly erroneous nature of "inconclusives" that needs to be fixed. More fundamentally, we argue that attempts to score "inconclusives" as errors amount to philosophical claims disguised as forensic methodology. Specifically, these attempts interfere with the metaphysical substrate underpinning empirical research. We point this out on the basis of the law of the excluded middle, i.e. the principle of "no third possibility being given" (tertium non datur).

Digital evidence exceptionalism? A review and discussion of conceptual hurdles in digital evidence transformation.

Forensic science is currently undergoing a transformation and expansion to include modern types of evidence, such as evidence generated by digital investigations. This development is said to raise a series of challenges, both in operational and conceptual dimensions. This paper reviews and discusses a series of convoluted conceptual hurdles that are encountered in connection with the use of digital evidence as part of evidence and proof processes at trial, in contradistinction to investigative uses of such types of evidence. As a recent example raising such hurdles, we analyse and discuss assertions and proposals made in the article "Digital Evidence Certainty Descriptors (DECDs)" by Graeme Horsman (32 Forensic Science International: Digital Investigation (2020) 200896).

Letter to the Editor: Commentary on "Is it possible to predict the origin of epithelial cells? - A comparison of secondary transfer of skin epithelial cells versus vaginal mucous membrane cells by direct contact, M.M. Bouzga et al., Science &Justice, https://doi.org/10.1016/j.scijus.2020.02.003".

This letter to the Editor comments on the paper "Is it possible to predict the origin of epithelial cells? - A comparison of secondary transfer of skin epithelial cells versus vaginal mucous membrane cells by direct contact" by M.M. Bouzga et al. (Science &Justice,https://doi.org/10.1016/j.scijus.20al.20.02.003).

Letter to the editor: Commentary on "Strategic choice in linear sequential unmasking, Roger Koppl, Science & Justice, https://doi.org/10.1016/j.scijus.2018.10.010".

This letter to the Editor comments on the paper 'Strategic choice in linear sequential unmasking' by Roger Koppl (Science & Justice, https://doi.org/10.1016/j.scijus.2018.10.010).

Are Inconclusive Decisions in Forensic Science as Deficient as They Are Said to Be?

Many quarters of forensic science use reporting formats such as "identification," "inconclusive," and "exclusion." These types of conclusions express opinions as to whether or not a particular person or object is the source of the material or traces of unknown source that is of interest in a given case. Rendering an "inconclusive" conclusion is sometimes criticized as being inadequate because-supposedly-it does not provide recipients of expert information with helpful directions. In this paper, we critically examine this claim using decision theory. We present and defend the viewpoint according to which deciding to render an "inconclusive" conclusion is, on a formal account, not as inadequate as may commonly be thought. Using elements of decision theory from existing accounts on the topic, we show that inconclusive conclusions can actually be viable alternatives with respect to other types of conclusions, such as "identification."

Decisional Dimensions in Expert Witness Testimony - A Structural Analysis.

The relationship between forensic science and legal adjudication is intricate mainly because the need to inform fact-finders on issues going beyond the layman's knowledge poses challenges both on empirical and normative dimensions, in particular with regards to the specific role and duties of the different participants in the legal process. While rationality is widely upheld as one of the aspirations of the legal process across many modern jurisdictions, a pending question is how to remedy the uneasy relationship between general propositions (and knowledge claims) conditioning expert witness testimony, and individualized decisions taken by fact-finders. The focus has hitherto been put on the utilization of model-based and formal methods of reasoning while, regrettably, the concepts of judgment and decision-making have not received equal attention. A first aspiration of our paper will thus be to further clarify the nature of this systemic relationship in the particular area of the legal process involving scientific experts, by conducting a critical transversal analysis of current empirical, normative and doctrinal understandings of expert witness testimony. As a second aim, we will use this insight to argue in favor of the view that structural features of expert witness testimony are embedded in a decision-making process, and that the understanding of this decisional dimension is important for clarifying the respective roles of expert witnesses and fact-finders, and for favoring their mutual understanding thereof. To substantiate this perspective, and attest to its growing recognition as a frontier understanding, we will provide real-world examples from forensic science reporting practice and policy documents of professional bodies.

Evaluation of forensic genetics findings given activity level propositions: A review.

The evaluation of results of forensic genetic analyses given activity level propositions is an emerging discipline in forensic genetics. Although it is a topic with a long history, it has never been considered to be such a critically important topic for the field, as today. With the increasing sensitivity of analysis techniques, and advances in data interpretation using probabilistic models ('probabilistic genotyping'), there is an increasing demand on forensic biologists to share specialised knowledge to help recipients of expert information address mode and timing of transfer and persistence of traces in court. Scientists thereby have a critical role in the assessment of their findings in the context of the case. This helps the judiciary with activity level inferences in a balanced, robust and transparent way, when based on (1) proper case assessment and interpretation respecting the hierarchy of propositions (supported by, for example, the use of Bayesian networks as graphical models), (2) use of appropriate data to inform probabilities, and (3) reporting guidelines by international bodies. This critical review of current literature shows that with certain prerequisites for training and quality assurance, there is a solid foundation for evidence interpretation when propositions of interest are at the 'activity level'.

A template for constructing Bayesian networks in forensic biology cases when considering activity level propositions.

The hierarchy of propositions has been accepted amongst the forensic science community for some time. It is also accepted that the higher up the hierarchy the propositions are, against which the scientist are competent to evaluate their results, the more directly useful the testimony will be to the court. Because each case represents a unique set of circumstances and findings, it is difficult to come up with a standard structure for evaluation. One common tool that assists in this task is Bayesian networks (BNs). There is much diversity in the way that BN can be constructed. In this work, we develop a template for BN construction that allows sufficient flexibility to address most cases, but enough commonality and structure that the flow of information in the BN is readily recognised at a glance. We provide seven steps that can be used to construct BNs within this structure and demonstrate how they can be applied, using a case example.

Reconsiderando la individualización forense como una decisión / Rethinking forensic individualization as a decision

No disponible

Towards a Bayesian evaluation of features in questioned handwritten signatures.

In this work, we propose the construction of a evaluative framework for supporting experts in questioned signature examinations. Through the use of Bayesian networks, we envision to quantify the probative value of well defined measurements performed on questioned signatures, in a way that is both formalised and part of a coherent approach to evaluation. At the current stage, our project is explorative, focusing on the broad range of aspects that relate to comparative signature examinations. The goal is to identify writing features which are both highly discriminant, and easy for forensic examiners to detect. We also seek for a balance between case-specific features and characteristics which can be measured in the vast majority of signatures. Care is also taken at preserving the interpretability at every step of the reasoning process. This paves the way for future work, which will aim at merging the different contributions to a single probabilistic measure of strength of evidence using Bayesian networks.

Helping to distinguish primary from secondary transfer events for trace DNA.

DNA is routinely recovered in criminal investigations. The sensitivity of laboratory equipment and DNA profiling kits means that it is possible to generate DNA profiles from very small amounts of cellular material. As a consequence, it has been shown that DNA we detect may not have arisen from a direct contact with an item, but rather through one or more intermediaries. Naturally the questions arising in court, particularly when considering trace DNA, are of how DNA may have come to be on an item. While scientists cannot directly answer this question, forensic biological results can help in discriminating between alleged activities. Much experimental research has been published showing the transfer and persistence of DNA under varying conditions, but as of yet the results of these studies have not been combined to deal with broad questions about transfer mechanisms. In this work we use published data and Bayesian networks to develop a statistical logical framework by which questions of transfer mechanism can be approached probabilistically. We also identify a number of areas where further work could be carried out in order to improve our knowledge base when helping to address questions about transfer mechanisms. Finally, we apply the constructed Bayesian network to ground truth known data to determine if, with current knowledge, there is any power in DNA quantities to distinguish primary and secondary transfer events.

Evaluation of Forensic DNA Traces When Propositions of Interest Relate to Activities: Analysis and Discussion of Recurrent Concerns.

When forensic scientists evaluate and report on the probative strength of single DNA traces, they commonly rely on only one number, expressing the rarity of the DNA profile in the population of interest. This is so because the focus is on propositions regarding the source of the recovered trace material, such as "the person of interest is the source of the crime stain." In particular, when the alternative proposition is "an unknown person is the source of the crime stain," one is directed to think about the rarity of the profile. However, in the era of DNA profiling technology capable of producing results from small quantities of trace material (i.e., non-visible staining) that is subject to easy and ubiquitous modes of transfer, the issue of source is becoming less central, to the point that it is often not contested. There is now a shift from the question "whose DNA is this?" to the question "how did it get there?" As a consequence, recipients of expert information are now very much in need of assistance with the evaluation of the meaning and probative strength of DNA profiling results when the competing propositions of interest refer to different activities. This need is widely demonstrated in day-to-day forensic practice and is also voiced in specialized literature. Yet many forensic scientists remain reluctant to assess their results given propositions that relate to different activities. Some scientists consider evaluations beyond the issue of source as being overly speculative, because of the lack of relevant data and knowledge regarding phenomena and mechanisms of transfer, persistence and background of DNA. Similarly, encouragements to deal with these activity issues, expressed in a recently released European guideline on evaluative reporting (Willis et al., 2015), which highlights the need for rethinking current practice, are sometimes viewed skeptically or are not considered feasible. In this discussion paper, we select and discuss recurrent skeptical views brought to our attention, as well as some of the alternative solutions that have been suggested. We will argue that the way forward is to address now, rather than later, the challenges associated with the evaluation of DNA results (from small quantities of trace material) in light of different activities to prevent them being misrepresented in court.

Discussion on how to implement a verbal scale in a forensic laboratory: Benefits, pitfalls and suggestions to avoid misunderstandings.

In a recently published guideline for evaluative reporting in forensic science, the European Network of Forensic Science Institutes (ENFSI) recommended the use of the likelihood ratio for the measurement of the value of forensic results. As a device to communicate the probative value of the results, the ENFSI guideline mentions the possibility to define and use a verbal scale, which should be unified within a forensic institution. This paper summarizes discussions held between scientists of our institution to develop and implement such a verbal scale. It intends to contribute to general discussions likely to be faced by any forensic institution that engages in continuous monitoring and improving of their evaluation and reporting format. We first present published arguments in favour of the use of such verbal qualifiers. We emphasise that verbal qualifiers do not replace the use of numbers to evaluate forensic findings, but are useful to communicate the probative value, since the weight of evidence in terms of likelihood ratio are still apprehended with difficulty by both the forensic scientists, especially in the absence of hard data, and the recipient of information. We further present arguments that support the development of the verbal scale that we propose. Recognising the limits of the use of such a verbal scale, we then discuss its disadvantages: it may lead to the spurious view according to which the value of the observations made in a given case is relative to other cases. Verbal qualifiers are also prone to misunderstandings and cannot be coherently combined with other evidence. We therefore recommend not using the verbal qualifier alone in a written statement. While scientists should only report on the probability of the findings - and not on the probability of the propositions, which are the duty of the Court - we suggest showing examples to let the recipient of information understand how the scientific evidence affects the probabilities of the propositions. To avoid misunderstandings, we also advise to mention in the statement what the results do not mean. Finally, we are of the opinion that if experts were able to coherently articulate numbers, and if recipients of information could properly handle such numbers, then verbal qualifiers could be abandoned completely. At that time, numerical expressions of probative value will be appropriately understood, as other numerical measures that most of us understand without the need of any further explanation, such as expressions for length or temperature.