Predicting suitability of finger marks using machine learning techniques and examiner annotations.

Previous research has established the variability of examiners in reaching suitability determinations for friction ridge comparisons. Attempts to create predictive models to assist in this determination have been made, but have been largely confined to fully automated processes that focus on suitability for AFIS entry. This work develops, optimizes, and validates a hybrid predictive model that utilizes both examiner-observed variables and automated measures of quality and rarity to arrive at suitability classifications along four scales that have been proposed in our previous research: Value, Complexity, AFIS, and Difficulty. We show that a model based only on automatically extracted quality or selectivity measures does not perform as well as when used in conjunction with a limited set of user inputs. The model is then based on a limited set of input from the users while taking advantage of automatic measures with a view to limit the user encoding effort while maintaining accuracy. The developed model is able to make predictions at up to 83.13% accuracy when using full study data and maintains similar levels of accuracy in an external validation study. The model achieved accuracy at a similar level to that of examiners asked to make the same suitability determinations across all scales. The model can easily be introduced into an operational laboratory with very little additional operational burden to provide guidance on suitability, complexity, AFIS, and quality assurance decisions; to assist in designing testing and training exercises of progressive difficulty; to describe the difficulty of a mark in testimony; and to provide a consensus-based opinion in laboratories where a second opinion is desired but the laboratory lacks sufficient personnel to form a consensus panel.

Automatic assessment of fingermarks quality: Exploration of the possible application in the context of detection and comparison with human examiners.

In forensic science, particularly in the context of latent fingermarks detection, forensic scientists are often faced with the need to assess the quality of the detected fingermarks to quantitatively interpret their results and express conclusions. Today this process is mainly carried out by human examiners referring to guidelines or provided quality scales. The largest the set of fingermarks (e.g., hundreds, thousands), the longest and the most labor-intensive this task becomes. Moreover, it is difficult to guarantee a fully objective process since the subjectivity of each individual is almost impossible to avoid, especially with regards to the interpretation of the quality scale levels or when facing fingermarks detected in an inhomogeneous manner. In this paper, the possibility of automatizing the quality assessment step is explored. The choice has been made to consider the use of quality assessment algorithms currently applied in an identification context. 150 natural fingermarks from ten donors were deposited on three different supports. These marks were detected using 1,2-indanedione/zinc or cyanoacrylate fumigation depending on the support. Then, their quality was assessed by five examiners, according to the UNIL scale, and by seven algorithms (i.e., Lights Out, Latent Fingerprint Image Quality 1 and 2, Latent Quality Metric, Expected Score Likelihood Ratio, NIST Fingerprint Image Quality, MINDTCT). Spearman and Pearson correlations were calculated, and the distribution of scores for each algorithm was charted (using boxplots) against the results provided by the human examiners. The most promising results were obtained with the LQM algorithm, more specifically with the fingermark clarity metric.

Mind-set - How bias leads to errors in friction ridge comparisons.

Mind-set is a term used in the friction ridge discipline to describe a confirmation bias in which an examiner makes early decisions about their interpretation of a mark but fails to update or reconsider those decisions in light of additional information. This most often occurs during the analysis of a mark when an examiner makes decisions (such as orientation or anatomical source of a mark) to help expedite a manual search or set parameters for an automated search, but fails to re-evaluate these decisions if the initial screening of available exemplars does not yield a comparable area, potentially leading to a miss or an erroneous exclusion. Mind-set can also occur when an examiner believes a comparison may be an identification early in the comparison process and employs poor comparison habits to convince themselves it is true, often creating or adapting comparison notes after seeing the exemplar, straining logic to justify their decision, and potentially leading to an erroneous identification. A recent black box study on palmar comparison accuracy and reliability noted both behaviors in the annotations and notes provided by some study participants. Examples are provided in this paper to serve as a reminder to examiners to not allow mind-set to lead them into errors. Particularly given the high false negative error rates reported throughout the literature, examiners need to make re-considering their initial analysis before rendering an exclusion decision part of their comparison routine.

Testing the accuracy and reliability of palmar friction ridge comparisons - A black box study.

Critics and commentators have been calling for some time for black box studies in the forensic science disciplines to establish the foundational validity of those fields-that is, to establish a discipline-wide, base-rate estimate of the error rates that may be expected in each field. While the well-known FBI/Noblis black box study has answered that call for fingerprints, no research to establish similar error rates for palmar impressions has been previously undertaken. We report the results of the first large-scale black box study to establish a discipline-wide error rate estimate for palmar comparisons. The 226 latent print examiner participants returned 12,279 decisions over a dataset of 526 known ground-truth pairings. There were 12 false identification decisions made yielding a false positive error rate of 0.7%. There were also 552 false exclusion decisions made yielding a false negative error rate of 9.5%. Given their larger number, false negative error rates were further stratified by size, comparison difficulty, and area of the palm from which the mark originated. The notion of "questionable conclusions," in which the ground truth response may not be the most appropriate, is introduced and discussed in light of the data obtained in the study. Measures of examiner consistency in analysis and comparison decisions are presented along with statistical analysis of the ability of many variables, such as demographics or image quality, to predict outcomes. Two online apps are introduced that will allow the reader to fully explore the results on their own, or to explore the notions of frequentist confidence intervals and Bayesian credible intervals.

Occurrence and associative value of non-identifiable fingermarks.

Fingermarks that have insufficient characteristics for identification often have discernible characteristics that could form the basis for lesser degrees of correspondence or probability of occurrence within a population. Currently, those latent prints that experts judge to be insufficient for identification are not used as associative evidence. How often do such prints occur and what is their potential value for association? The answers are important. We could be routinely setting aside a very important source of associative evidence, with high potential impact, in many cases; or such prints might be of very low utility, adding very little, or only very rarely contributing to cases in a meaningful way. The first step is to better understand the occurrence and range of associative value of these fingermarks. The project goal was to explore and test a theory that in large numbers of cases fingermarks of no value for identification purposes occur and are readily available, though not used, and yet have associative value that could provide useful information. Latent fingermarks were collected from nine state and local jurisdictions. Fingermarks included were those (1) collected in the course of investigations using existing jurisdictional procedures, (2) originally assessed by the laboratory as of no value for identification (NVID), (3) re-assessed by expert review as NVID, but with least three clear and reliable minutiae in relationship to one another, and (4) determined to show at least three auto-encoded minutiae. An expected associative value (ESLR) for each mark was measured, without reference to a putative source, based on modeling within-variability and between-variability of AFIS scores. This method incorporated (1) latest generation feature extraction, (2) a (minutiae-only) matcher, (3) a validated distortion model, and (4) NIST SD27 database calibration. Observed associative value distributions were determined for violent crimes, property crimes, and for existing objective measurements of latent print quality. 750 Non Identifiable Fingermarks (NIFMs) showed values of Log10 ESLR ranging from 1.05-10.88, with a mean value of 5.56 (s.d. 2.29), corresponding to an ESLR of approximately 380,000. It is clear that there are large numbers of cases where NIFMs occur that have high potential associative value as indicated by the ESLR. These NIFMs are readily available, but not used, yet have associative value that could provide useful information. These findings lead to the follow-on questions, "How useful would NIFM evidence be in actual practice?" and, "What developments or improvements are needed to maximize this contribution?"