From data to a validated score-based LR system: A practitioner's guide.

Likelihood ratios (LRs) are a useful measure of evidential strength. In forensic casework consisting of a flow of cases with essentially the same question and the same analysis method, it is feasible to construct an 'LR system', that is, an automated procedure that has the observations as input and an LR as output. This paper is aimed at practitioners interested in building their own LR systems. It gives an overview of the different steps needed to get to a validated LR system from data. The paper is accompanied by a notebook that illustrates each step with an example using glass data. The notebook introduces open-source software in Python constructed by NFI (Netherlands Forensic Institute) data scientists and statisticians.

Development and validation of a fast and automated DNA identification line.

The importance of DNA evidence for gaining investigative leads demands a fast workflow for forensic DNA profiling performed in large volumes. Therefore, we developed software solutions for automated DNA profile analysis, contamination check, major donor inference, DNA database (DDB) comparison and reporting of the conclusions. This represents the Fast DNA IDentification Line (FIDL) and this study describes its development, validation and implementation in criminal casework at the authors' institute. This first implementation regards single donor profiles and major contributors to mixtures. The validation included testing of the software components on their own and examination of the performance of different DDB search strategies. Furthermore, end-to-end testing was performed under three conditions: (1) testing of scenarios that can occur in DNA casework practice, (2) tests using three months of previous casework data, and (3) testing in a casework production environment in parallel to standard casework practices. The same DNA database candidates were retrieved by this automated line as by the manual workflow. The data flow was correct, results were reproducible and robust, results requiring manual analysis were correctly flagged, and reported results were as expected. Overall, we found FIDL valid for use in casework practice in our institute. The results from FIDL are automatically reported within three working days from receiving the trace sample. This includes the time needed for registration of the case, DNA extraction, quantification, polymerase chain reaction and capillary electrophoresis. FIDL itself takes less than two hours from intake of the raw CE data to reporting. Reported conclusions are one of five options: (1) candidate retrieved from DDB, (2) no candidate retrieved from DDB, (3) high evidential value with regards to reference within the case, (4) results require examination of expert, or (5) insufficient amount of DNA obtained to generate a DNA profile. In our current process, the automated report is sent within three working days and a complete report, with confirmation of the FIDL results, and signed by a reporting officer is sent at a later time. The signed report may include additional analyses regarding e.g. minor contributors. The automated report with first case results is quickly available to the police enabling them to act upon the DNA results prior to receiving the full DNA report. This line enables a uniform and efficient manner of handling large numbers of traces and cases and provides high value investigative leads in the early stages of the investigation.

Explainable artificial intelligence in forensics: Realistic explanations for number of contributor predictions of DNA profiles.

Machine learning obtains good accuracy in determining the number of contributors (NOC) in short tandem repeat (STR) mixture DNA profiles. However, the models used so far are not understandable to users as they only output a prediction without any reasoning for that conclusion. Therefore, we leverage techniques from the field of explainable artificial intelligence (XAI) to help users understand why specific predictions are made. Where previous attempts at explainability for NOC estimation have relied upon using simpler, more understandable models that achieve lower accuracy, we use techniques that can be applied to any machine learning model. Our explanations incorporate SHAP values and counterfactual examples for each prediction into a single visualization. Existing methods for generating counterfactuals focus on uncorrelated features. This makes them inappropriate for the highly correlated features derived from STR data for NOC estimation, as these techniques simulate combinations of features that could not have resulted from an STR profile. For this reason, we have constructed a new counterfactual method, Realistic Counterfactuals (ReCo), which generates realistic counterfactual explanations for correlated data. We show that ReCo outperforms state-of-the-art methods on traditional metrics, as well as on a novel realism score. A user evaluation of the visualization shows positive opinions of end-users, which is ultimately the most appropriate metric in assessing explanations for real-world settings.

Consensus on validation of forensic voice comparison.

Since the 1960s, there have been calls for forensic voice comparison to be empirically validated under casework conditions. Since around 2000, there have been an increasing number of researchers and practitioners who conduct forensic-voice-comparison research and casework within the likelihood-ratio framework. In recent years, this community of researchers and practitioners has made substantial progress toward validation under casework conditions becoming a standard part of practice: Procedures for conducting validation have been developed, along with graphics and metrics for representing the results, and an increasing number of papers are being published that include empirical validation of forensic-voice-comparison systems under conditions reflecting casework conditions. An outstanding question, however, is: In the context of a case, given the results of an empirical validation of a forensic-voice-comparison system, how can one decide whether the system is good enough for its output to be used in court? This paper provides a statement of consensus developed in response to this question. Contributors included individuals who had knowledge and experience of validating forensic-voice-comparison systems in research and/or casework contexts, and individuals who had actually presented validation results to courts. They also included individuals who could bring a legal perspective on these matters, and individuals with knowledge and experience of validation in forensic science more broadly. We provide recommendations on what practitioners should do when conducting evaluations and validations, and what they should present to the court. Although our focus is explicitly on forensic voice comparison, we hope that this contribution will be of interest to an audience concerned with validation in forensic science more broadly. Although not written specifically for a legal audience, we hope that this contribution will still be of interest to lawyers.

Transmission tree of the highly pathogenic avian influenza (H5N1) epidemic in Israel, 2015.

The transmission tree of the Israeli 2015 epidemic of highly pathogenic avian influenza (H5N1) was modelled by combining the spatio-temporal distribution of the outbreaks and the genetic distance between virus isolates. The most likely successions of transmission events were determined and transmission parameters were estimated. It was found that the median infectious pressure exerted at 1 km was 1.59 times (95% CI 1.04, 6.01) and 3.54 times (95% CI 1.09, 131.75) higher than that exerted at 2 and 5 km, respectively, and that three farms were responsible for all seven transmission events.

Statistical Analysis of Tract-Tracing Experiments Demonstrates a Dense, Complex Cortical Network in the Mouse.

Anatomical tract tracing methods are the gold standard for estimating the weight of axonal connectivity between a pair of pre-defined brain regions. Large studies, comprising hundreds of experiments, have become feasible by automated methods. However, this comes at the cost of positive-mean noise making it difficult to detect weak connections, which are of particular interest as recent high resolution tract-tracing studies of the macaque have identified many more weak connections, adding up to greater connection density of cortical networks, than previously recognized. We propose a statistical framework that estimates connectivity weights and credibility intervals from multiple tract-tracing experiments. We model the observed signal as a log-normal distribution generated by a combination of tracer fluorescence and positive-mean noise, also accounting for injections into multiple regions. Using anterograde viral tract-tracing data provided by the Allen Institute for Brain Sciences, we estimate the connection density of the mouse intra-hemispheric cortical network to be 73% (95% credibility interval (CI): 71%, 75%); higher than previous estimates (40%). Inter-hemispheric density was estimated to be 59% (95% CI: 54%, 62%). The weakest estimable connections (about 6 orders of magnitude weaker than the strongest connections) are likely to represent only one or a few axons. These extremely weak connections are topologically more random and longer distance than the strongest connections, which are topologically more clustered and shorter distance (spatially clustered). Weak links do not substantially contribute to the global topology of a weighted brain graph, but incrementally increased topological integration of a binary graph. The topology of weak anatomical connections in the mouse brain, rigorously estimable down to the biological limit of a single axon between cortical areas in these data, suggests that they might confer functional advantages for integrative information processing and/or they might represent a stochastic factor in the development of the mouse connectome.

Default Mode Hypoconnectivity Underlies a Sex-Related Autism Spectrum.

BACKGROUND: Females and males differ significantly in the prevalence and presentation of autism spectrum conditions. One theory of this effect postulates that autistic traits lie on a sex-related continuum in the general population, and autism represents the extreme male end of this spectrum. This theory predicts that any feature of autism in males should 1) be present in autistic females, 2) differentiate between the sexes in the typical population, and 3) correlate with autistic traits. We tested these three predictions for default mode network (DMN) hypoconnectivity during the resting state, one of the most robustly found neurobiological differences in autism. METHODS: We analyzed a primary dataset of adolescents (N = 121, 12-18 years of age) containing a relatively large number of females and a replication multisite dataset including children, adolescents, and adults (N = 980, 6-58 years of age). We quantified the average connectivity between DMN regions and tested for group differences and correlation with behavioral performance using robust regression. RESULTS: We found significant differences in DMN intraconnectivity between female controls and females with autism (p = .001 in the primary dataset; p = .009 in the replication dataset), and between female controls and male controls (p = .036 in the primary dataset; p = .002 in the replication dataset). We also found a significant correlation between DMN intraconnectivity and performance on a mentalizing task (p = .001) in the primary dataset. CONCLUSIONS: Collectively, these findings provide the first evidence for DMN hypoconnectivity as a behaviorally relevant neuroimaging phenotype of the sex-related spectrum of autistic traits, of which autism represents the extreme case.

Graph theory analysis of complex brain networks: new concepts in brain mapping applied to neurosurgery.

Neuroanatomy has entered a new era, culminating in the search for the connectome, otherwise known as the brain's wiring diagram. While this approach has led to landmark discoveries in neuroscience, potential neurosurgical applications and collaborations have been lagging. In this article, the authors describe the ideas and concepts behind the connectome and its analysis with graph theory. Following this they then describe how to form a connectome using resting state functional MRI data as an example. Next they highlight selected insights into healthy brain function that have been derived from connectome analysis and illustrate how studies into normal development, cognitive function, and the effects of synthetic lesioning can be relevant to neurosurgery. Finally, they provide a précis of early applications of the connectome and related techniques to traumatic brain injury, functional neurosurgery, and neurooncology.

Wiring cost and topological participation of the mouse brain connectome.

Brain connectomes are topologically complex systems, anatomically embedded in 3D space. Anatomical conservation of "wiring cost" explains many but not all aspects of these networks. Here, we examined the relationship between topology and wiring cost in the mouse connectome by using data from 461 systematically acquired anterograde-tracer injections into the right cortical and subcortical regions of the mouse brain. We estimated brain-wide weights, distances, and wiring costs of axonal projections and performed a multiscale topological and spatial analysis of the resulting weighted and directed mouse brain connectome. Our analysis showed that the mouse connectome has small-world properties, a hierarchical modular structure, and greater-than-minimal wiring costs. High-participation hubs of this connectome mediated communication between functionally specialized and anatomically localized modules, had especially high wiring costs, and closely corresponded to regions of the default mode network. Analyses of independently acquired histological and gene-expression data showed that nodal participation colocalized with low neuronal density and high expression of genes enriched for cognition, learning and memory, and behavior. The mouse connectome contains high-participation hubs, which are not explained by wiring-cost minimization but instead reflect competitive selection pressures for integrated network topology as a basis for higher cognitive and behavioral functions.

Relating phylogenetic trees to transmission trees of infectious disease outbreaks.

Transmission events are the fundamental building blocks of the dynamics of any infectious disease. Much about the epidemiology of a disease can be learned when these individual transmission events are known or can be estimated. Such estimations are difficult and generally feasible only when detailed epidemiological data are available. The genealogy estimated from genetic sequences of sampled pathogens is another rich source of information on transmission history. Optimal inference of transmission events calls for the combination of genetic data and epidemiological data into one joint analysis. A key difficulty is that the transmission tree, which describes the transmission events between infected hosts, differs from the phylogenetic tree, which describes the ancestral relationships between pathogens sampled from these hosts. The trees differ both in timing of the internal nodes and in topology. These differences become more pronounced when a higher fraction of infected hosts is sampled. We show how the phylogenetic tree of sampled pathogens is related to the transmission tree of an outbreak of an infectious disease, by the within-host dynamics of pathogens. We provide a statistical framework to infer key epidemiological and mutational parameters by simultaneously estimating the phylogenetic tree and the transmission tree. We test the approach using simulations and illustrate its use on an outbreak of foot-and-mouth disease. The approach unifies existing methods in the emerging field of phylodynamics with transmission tree reconstruction methods that are used in infectious disease epidemiology.

Viral substitution rate variation can arise from the interplay between within-host and epidemiological dynamics.

The evolutionary rates of RNA viruses can differ from one another by several orders of magnitude. Much of this variation has been explained by differences in viral mutation rates and selective environments. However, substitution rates also vary considerably across viral populations belonging to the same species. In particular, viral lineages from epidemic regions tend to have higher substitution rates than those from endemic regions, and lineages from populations with higher contact rates tend to have higher substitution rates than those from populations with lower contact rates. We address the mechanism behind these patterns by using a nested modeling approach, whereby we integrate within-host viral replication dynamics with a population-level epidemiological model. Through numerical simulations and analytical approximations, we show that variation in viral substitution rates over the course of an infection, coupled with differences in age of infection of transmitting hosts under different epidemiological scenarios, can explain these evolutionary patterns. We further derive analytical estimates of expected substitution rate differences under epidemic versus endemic epidemiological conditions. By comparing these estimates to empirical data for four viral species, we show that these factors are sufficient to explain observed variation in substitution rates in three of four cases. This work shows that even in neutrally evolving viral populations, epidemiological dynamics can alter substitution rates via the interplay between within-host replication dynamics and population-level disease dynamics.

Finding evidence for local transmission of contagious disease in molecular epidemiological datasets.

Surveillance systems of contagious diseases record information on cases to monitor incidence of disease and to evaluate effectiveness of interventions. These systems focus on a well-defined population; a key question is whether observed cases are infected through local transmission within the population or whether cases are the result of importation of infection into the population. Local spread of infection calls for different intervention measures than importation of infection. Besides standardized information on time of symptom onset and location of cases, pathogen genotyping or sequencing offers essential information to address this question. Here we introduce a method that takes full advantage of both the genetic and epidemiological data to distinguish local transmission from importation of infection, by comparing inter-case distances in temporal, spatial and genetic data. Cases that are part of a local transmission chain will have shorter distances between their geographical locations, shorter durations between their times of symptom onset and shorter genetic distances between their pathogen sequences as compared to cases that are due to importation. In contrast to generic clustering algorithms, the proposed method explicitly accounts for the fact that during local transmission of a contagious disease the cases are caused by other cases. No pathogen-specific assumptions are needed due to the use of ordinal distances, which allow for direct comparison between the disparate data types. Using simulations, we test the performance of the method in identifying local transmission of disease in large datasets, and assess how sensitivity and specificity change with varying size of local transmission chains and varying overall disease incidence.

A sign of superspreading in tuberculosis: highly skewed distribution of genotypic cluster sizes.

BACKGROUND: Molecular typing is a valuable tool for gaining insight into spread of Mycobacterium tuberculosis. Typing allows for clustering of cases whose isolates share an identical genotype, revealing epidemiologic relatedness. Observed distributions of genotypic cluster sizes of tuberculosis (TB) are highly skewed. A possible explanation for this skewness is the concept of "superspreading": a high heterogeneity in the number of secondary cases caused per infectious individual. Superspreading has been previously found for diseases such as severe acute respiratory syndrome and smallpox, where the entire transmission tree is known. So far, no method exists to relate superspreading to the distribution of genotypic cluster sizes. METHODS: We quantified heterogeneity in secondary infections per infectious individual by describing this number as a negative binomial distribution. The dispersion parameter k is a measure of superspreading; standard (homogeneous) models use values of k ≥ 1, whereas small values of k imply superspreading. We estimated this negative binomial dispersion parameter for TB in the Netherlands, using the genotypic cluster size distribution for all 8330 cases of culture confirmed, pulmonary TB diagnosed between 1993 and 2007 in the Netherlands. RESULTS: The dispersion parameter k was estimated at 0.10 (95% confidence interval = 0.09-0.12), well in the range of values consistent with superspreading. Simulation studies showed the method reliably estimates the dispersion parameter across a range of scenarios and parameter values. CONCLUSION: Heterogeneity in the number of secondary cases caused per infectious individual is a plausible explanation for the observed skewness in genotypic cluster size distribution of TB.

Genetic data provide evidence for wind-mediated transmission of highly pathogenic avian influenza.

Outbreaks of highly pathogenic avian influenza in poultry can cause severe economic damage and represent a public health threat. Development of efficient containment measures requires an understanding of how these influenza viruses are transmitted between farms. However, the actual mechanisms of interfarm transmission are largely unknown. Dispersal of infectious material by wind has been suggested, but never demonstrated, as a possible cause of transmission between farms. Here we provide statistical evidence that the direction of spread of avian influenza A(H7N7) is correlated with the direction of wind at date of infection. Using detailed genetic and epidemiological data, we found the direction of spread by reconstructing the transmission tree for a large outbreak in the Netherlands in 2003. We conservatively estimate the contribution of a possible wind-mediated mechanism to the total amount of spread during this outbreak to be around 18%.