Critical assessment of variant prioritization methods for rare disease diagnosis within the rare genomes project.

BACKGROUND: A major obstacle faced by families with rare diseases is obtaining a genetic diagnosis. The average "diagnostic odyssey" lasts over five years and causal variants are identified in under 50%, even when capturing variants genome-wide. To aid in the interpretation and prioritization of the vast number of variants detected, computational methods are proliferating. Knowing which tools are most effective remains unclear. To evaluate the performance of computational methods, and to encourage innovation in method development, we designed a Critical Assessment of Genome Interpretation (CAGI) community challenge to place variant prioritization models head-to-head in a real-life clinical diagnostic setting. METHODS: We utilized genome sequencing (GS) data from families sequenced in the Rare Genomes Project (RGP), a direct-to-participant research study on the utility of GS for rare disease diagnosis and gene discovery. Challenge predictors were provided with a dataset of variant calls and phenotype terms from 175 RGP individuals (65 families), including 35 solved training set families with causal variants specified, and 30 unlabeled test set families (14 solved, 16 unsolved). We tasked teams to identify causal variants in as many families as possible. Predictors submitted variant predictions with estimated probability of causal relationship (EPCR) values. Model performance was determined by two metrics, a weighted score based on the rank position of causal variants, and the maximum F-measure, based on precision and recall of causal variants across all EPCR values. RESULTS: Sixteen teams submitted predictions from 52 models, some with manual review incorporated. Top performers recalled causal variants in up to 13 of 14 solved families within the top 5 ranked variants. Newly discovered diagnostic variants were returned to two previously unsolved families following confirmatory RNA sequencing, and two novel disease gene candidates were entered into Matchmaker Exchange. In one example, RNA sequencing demonstrated aberrant splicing due to a deep intronic indel in ASNS, identified in trans with a frameshift variant in an unsolved proband with phenotypes consistent with asparagine synthetase deficiency. CONCLUSIONS: Model methodology and performance was highly variable. Models weighing call quality, allele frequency, predicted deleteriousness, segregation, and phenotype were effective in identifying causal variants, and models open to phenotype expansion and non-coding variants were able to capture more difficult diagnoses and discover new diagnoses. Overall, computational models can significantly aid variant prioritization. For use in diagnostics, detailed review and conservative assessment of prioritized variants against established criteria is needed.

Critical assessment of variant prioritization methods for rare disease diagnosis within the Rare Genomes Project.

Background: A major obstacle faced by rare disease families is obtaining a genetic diagnosis. The average "diagnostic odyssey" lasts over five years, and causal variants are identified in under 50%. The Rare Genomes Project (RGP) is a direct-to-participant research study on the utility of genome sequencing (GS) for diagnosis and gene discovery. Families are consented for sharing of sequence and phenotype data with researchers, allowing development of a Critical Assessment of Genome Interpretation (CAGI) community challenge, placing variant prioritization models head-to-head in a real-life clinical diagnostic setting. Methods: Predictors were provided a dataset of phenotype terms and variant calls from GS of 175 RGP individuals (65 families), including 35 solved training set families, with causal variants specified, and 30 test set families (14 solved, 16 unsolved). The challenge tasked teams with identifying the causal variants in as many test set families as possible. Ranked variant predictions were submitted with estimated probability of causal relationship (EPCR) values. Model performance was determined by two metrics, a weighted score based on rank position of true positive causal variants and maximum F-measure, based on precision and recall of causal variants across EPCR thresholds. Results: Sixteen teams submitted predictions from 52 models, some with manual review incorporated. Top performing teams recalled the causal variants in up to 13 of 14 solved families by prioritizing high quality variant calls that were rare, predicted deleterious, segregating correctly, and consistent with reported phenotype. In unsolved families, newly discovered diagnostic variants were returned to two families following confirmatory RNA sequencing, and two prioritized novel disease gene candidates were entered into Matchmaker Exchange. In one example, RNA sequencing demonstrated aberrant splicing due to a deep intronic indel in ASNS, identified in trans with a frameshift variant, in an unsolved proband with phenotype overlap with asparagine synthetase deficiency. Conclusions: By objective assessment of variant predictions, we provide insights into current state-of-the-art algorithms and platforms for genome sequencing analysis for rare disease diagnosis and explore areas for future optimization. Identification of diagnostic variants in unsolved families promotes synergy between researchers with clinical and computational expertise as a means of advancing the field of clinical genome interpretation.

Advancing Understanding of Inequities in Rare Disease Genomics.

PURPOSE: Advances in genomic research have facilitated rare disease diagnosis for thousands of individuals. Unfortunately, the benefits of advanced genetic diagnostic technology are not distributed equitably among the population, as has been seen in many other health care contexts. Quantifying and describing inequities in genetic diagnostic yield is inherently challenging due to barriers to both clinical and research genetic testing. We therefore present an implementation protocol developed to expand access to our rare disease genomic research study and to further understand existing inequities. METHODS AND FINDINGS: The Rare Genomes Project (RGP) at the Broad Institute of MIT and Harvard offers research genome sequencing to individuals with rare disease who remain genetically undiagnosed through direct interaction with the individual or family. This presents an opportunity for diagnosis beyond the clinical context, thus eliminating many barriers to access. An initial goal of RGP was to equalize access to genomic sequencing by decoupling testing access from proximity to a major medical center and physician referral. However, study participants over the initial 3 years of this project were predominantly white and well resourced. To further understand and address the lack of diversity within RGP, we developed a novel protocol embedded within the larger RGP study, in an approach informed by an implementation science framework. The aims of this protocol were: (1) to diversify recruitment and enrollment within RGP; (2) understand the process and context of implementing genomic medicine for rare disease diagnosis; and (3) investigate the value of a diagnosis for underserved populations. IMPLICATIONS: Improved understanding of existing inequities and potential strategies to address them are needed to advance equity in rare disease genetic diagnosis and research. In addition to the moral imperative of equity in genomic medicine, this approach is critical in order to fully understand the genomic underpinnings of rare disease.

Advancing Understanding of Inequities in Rare Disease Genomics.

Purpose: Advances in genomic research have led to the diagnosis of rare, early-onset diseases for thousands of individuals. Unfortunately, the benefits of advanced genetic diagnostic technology are not distributed equitably among the population, as has been seen in many other healthcare contexts. Even quantifying and describing inequities in genetic diagnostic yield is challenging due to variation in referrals to clinical genetics practices and other barriers to clinical genetic testing. Methods: The Rare Genomes Project (RGP) at the Broad Institute of MIT and Harvard offers research genome sequencing to individuals with rare disease who remain genetically undiagnosed through direct interaction with the individual or family. This presents an opportunity for diagnosis beyond the clinical context, thus eliminating many barriers to access. Findings: An initial goal of RGP was to equalize access to genomic sequencing by decoupling testing access from proximity to a major medical center and physician referral. However, our study participants are overwhelmingly non-disadvantaged, as evidenced by their access to specialist care and genetic testing prior to RGP enrollment, and are also predominantly white. Implications: We therefore describe our novel initiative to diversify RGP enrollment in order to advance equity in rare disease genetic diagnosis and research. In addition to the moral imperative of medical equity, this is also critical in order to fully understand the genomic underpinnings of rare disease. We utilize a mixed methods approach to understand the priorities and values of underrepresented communities, existing disparities, and the obstacles to addressing them: all of which is necessary to promote equity in future genomic medicine initiatives.