Resource requirements to accelerate clinical applications of next generation sequencing and radiomics: Workshop commentary and review.

The National Institutes of Health (NIH)/U.S. Food and Drug Administration (FDA) Joint Leadership Council Next-Generation Sequencing (NGS) and Radiomics Working Group (NGS&R WG) was formed by the NIH/FDA Joint Leadership Council to promote the development and validation of innovative NGS tests, radiomic tools, and associated data analysis and interpretation enhanced by artificial intelligence (AI) and machine-learning (ML) technologies. A two-day workshop was held on September 29-30, 2021 to convene members of the scientific community to discuss how to overcome the "ground truth" gap that has frequently been acknowledged as one of the limiting factors impeding high-quality research, development, validation, and regulatory science in these fields. This report provides a summary of the resource gaps identified by the WG and attendees, highlights existing resources and the ways they can potentially be leveraged to accelerate growth in these fields, and presents opportunities to support NGS and radiomic tool development and validation using technologies such as AI and ML.

FDA Approval Summary: Selpercatinib for the Treatment of Advanced RET Fusion-Positive Solid Tumors.

On September 21, 2022, the FDA granted accelerated approval to selpercatinib (Retevmo, Eli Lilly and Company) for the treatment of adult patients with locally advanced or metastatic solid tumors with a rearranged during transfection (RET) gene fusion that have progressed on or following prior systemic treatment or who have no satisfactory alternative treatment options. The approval was based on data from Study LOXO-RET-17001 (LIBRETTO-001; NCT03157128), an international, non-randomized, multi-cohort clinical trial that included patients with advanced solid tumors harboring RET alterations. The overall response rate in 41 patients with locally advanced or metastatic RET fusion-positive solid tumors other than non-small cell lung cancer (NSCLC) or thyroid cancer was 44% [95% confidence interval (CI), 28%-60%], with median duration of response 24.5 months (95% CI, 9.2-not evaluable). Patients with 10 of 14 tumor types with a variety of fusion partners had objective responses, including patients with the following tumors: pancreatic adenocarcinoma, colorectal, salivary, unknown primary, breast, soft-tissue sarcoma, bronchial carcinoid, ovarian, small intestine, and cholangiocarcinoma. The recommendation for approval was supported by results from LIBRETTO-001 in patients with RET fusion-positive NSCLC and thyroid cancer, which formed the basis of prior approvals in these tumor types. The most common adverse reactions (>25%) were edema, diarrhea, fatigue, dry mouth, hypertension, abdominal pain, constipation, rash, nausea, and headache. This is the first tissue-agnostic approval of a RET-directed targeted therapy.

A community effort to identify and correct mislabeled samples in proteogenomic studies.

Sample mislabeling or misannotation has been a long-standing problem in scientific research, particularly prevalent in large-scale, multi-omic studies due to the complexity of multi-omic workflows. There exists an urgent need for implementing quality controls to automatically screen for and correct sample mislabels or misannotations in multi-omic studies. Here, we describe a crowdsourced precisionFDA NCI-CPTAC Multi-omics Enabled Sample Mislabeling Correction Challenge, which provides a framework for systematic benchmarking and evaluation of mislabel identification and correction methods for integrative proteogenomic studies. The challenge received a large number of submissions from domestic and international data scientists, with highly variable performance observed across the submitted methods. Post-challenge collaboration between the top-performing teams and the challenge organizers has created an open-source software, COSMO, with demonstrated high accuracy and robustness in mislabeling identification and correction in simulated and real multi-omic datasets.

Best practices for benchmarking germline small-variant calls in human genomes.

Standardized benchmarking approaches are required to assess the accuracy of variants called from sequence data. Although variant-calling tools and the metrics used to assess their performance continue to improve, important challenges remain. Here, as part of the Global Alliance for Genomics and Health (GA4GH), we present a benchmarking framework for variant calling. We provide guidance on how to match variant calls with different representations, define standard performance metrics, and stratify performance by variant type and genome context. We describe limitations of high-confidence calls and regions that can be used as truth sets (for example, single-nucleotide variant concordance of two methods is 99.7% inside versus 76.5% outside high-confidence regions). Our web-based app enables comparison of variant calls against truth sets to obtain a standardized performance report. Our approach has been piloted in the PrecisionFDA variant-calling challenges to identify the best-in-class variant-calling methods within high-confidence regions. Finally, we recommend a set of best practices for using our tools and evaluating the results.

Author Correction: Best practices for benchmarking germline small-variant calls in human genomes.

In the version of this article initially published online, two pairs of headings were switched with each other in Table 4: "Recall (PCR free)" was switched with "Recall (with PCR)," and "Precision (PCR free)" was switched with "Precision (with PCR)." The error has been corrected in the print, PDF and HTML versions of this article.

Principles and Recommendations for Standardizing the Use of the Next-Generation Sequencing Variant File in Clinical Settings.

A national workgroup convened by the Centers for Disease Control and Prevention identified principles and made recommendations for standardizing the description of sequence data contained within the variant file generated during the course of clinical next-generation sequence analysis for diagnosing human heritable conditions. The specifications for variant files were initially developed to be flexible with regard to content representation to support a variety of research applications. This flexibility permits variation with regard to how sequence findings are described and this depends, in part, on the conventions used. For clinical laboratory testing, this poses a problem because these differences can compromise the capability to compare sequence findings among laboratories to confirm results and to query databases to identify clinically relevant variants. To provide for a more consistent representation of sequence findings described within variant files, the workgroup made several recommendations that considered alignment to a common reference sequence, variant caller settings, use of genomic coordinates, and gene and variant naming conventions. These recommendations were considered with regard to the existing variant file specifications presently used in the clinical setting. Adoption of these recommendations is anticipated to reduce the potential for ambiguity in describing sequence findings and facilitate the sharing of genomic data among clinical laboratories and other entities.

A research roadmap for next-generation sequencing informatics.

Next-generation sequencing technologies are fueling a wave of new diagnostic tests. Progress on a key set of nine research challenge areas will help generate the knowledge required to advance effectively these diagnostics to the clinic.

The first FDA marketing authorizations of next-generation sequencing technology and tests: challenges, solutions and impact for future assays.

The rapid emergence and clinical translation of novel high-throughput sequencing technologies created a need to clarify the regulatory pathway for the evaluation and authorization of these unique technologies. Recently, the US FDA authorized for marketing four next generation sequencing (NGS)-based diagnostic devices which consisted of two heritable disease-specific assays, library preparation reagents and a NGS platform that are intended for human germline targeted sequencing from whole blood. These first authorizations can serve as a case study in how different types of NGS-based technology are reviewed by the FDA. In this manuscript we describe challenges associated with the evaluation of these novel technologies and provide an overview of what was reviewed. Besides making validated NGS-based devices available for in vitro diagnostic use, these first authorizations create a regulatory path for similar future instruments and assays.

Statistical design for biospecimen cohort size in proteomics-based biomarker discovery and verification studies.

Protein biomarkers are needed to deepen our understanding of cancer biology and to improve our ability to diagnose, monitor, and treat cancers. Important analytical and clinical hurdles must be overcome to allow the most promising protein biomarker candidates to advance into clinical validation studies. Although contemporary proteomics technologies support the measurement of large numbers of proteins in individual clinical specimens, sample throughput remains comparatively low. This problem is amplified in typical clinical proteomics research studies, which routinely suffer from a lack of proper experimental design, resulting in analysis of too few biospecimens to achieve adequate statistical power at each stage of a biomarker pipeline. To address this critical shortcoming, a joint workshop was held by the National Cancer Institute (NCI), National Heart, Lung, and Blood Institute (NHLBI), and American Association for Clinical Chemistry (AACC) with participation from the U.S. Food and Drug Administration (FDA). An important output from the workshop was a statistical framework for the design of biomarker discovery and verification studies. Herein, we describe the use of quantitative clinical judgments to set statistical criteria for clinical relevance and the development of an approach to calculate biospecimen sample size for proteomic studies in discovery and verification stages prior to clinical validation stage. This represents a first step toward building a consensus on quantitative criteria for statistical design of proteomics biomarker discovery and verification research.

Characterization of whole genome amplified (WGA) DNA for use in genotyping assay development.

BACKGROUND: Genotyping assays often require substantial amounts of DNA. To overcome the problem of limiting amounts of available DNA, Whole Genome Amplification (WGA) methods have been developed. The multiple displacement amplification (MDA) method using Φ29 polymerase has become the preferred choice due to its high processivity and low error rate. However, the uniformity and fidelity of the amplification process across the genome has not been extensively characterized. RESULTS: To assess amplification uniformity, we used array-based comparative genomic hybridization (aCGH) to evaluate DNA copy number variations (CNVs) in DNAs amplified by two MDA kits: GenomiPhi and REPLI-g. The Agilent Human CGH array containing nearly one million probes was used in this study together with DNAs from a normal subject and 2 cystic fibrosis (CF) patients. Each DNA sample was amplified 4 independent times and compared to its native unamplified DNA. Komogorov distances and Phi correlations showed a high consistency within each sample group. Less than 2% of the probes showed more than 2-fold CNV introduced by the amplification process. The two amplification kits, REPLI-g and GenomiPhi, generate very similar amplified DNA samples despite the differences between the unamplified and amplified DNA samples. The results from aCGH analysis indicated that there were no obvious CNVs in the CFTR gene region due to WGA when compared to unamplified DNA. This was confirmed by quantitative real-time PCR copy number assays at 10 locations within the CFTR gene. DNA sequencing analysis of a 2-kb region within the CFTR gene showed no mutations introduced by WGA. CONCLUSION: The relatively high uniformity and consistency of the WGA process, coupled with the low replication error rate, suggests that WGA DNA may be suitable for accurate genotyping. Regions of the genome that were consistently under-amplified were found to contain higher than average GC content. Because of the consistent differences between the WGA DNA and the native unamplified DNA, characterization of the genomic region of interest, as described here, will be necessary to ensure the reliability of genotyping results from WGA DNA.

Regulatory perspective on translating proteomic biomarkers to clinical diagnostics.

Issues associated with the translation of complex proteomic biomarkers from discovery to clinical diagnostics have been widely discussed among academic researchers, government agencies, as well as assay and instrumentation manufacturers. Here, we provide an overview of the regulatory framework and type of information that is typically required in order to evaluate in vitro diagnostic tests regulated by the Office of In Vitro Diagnostic Device Evaluation and Safety (OIVD) at the US Food and Drug Administration (FDA), with the focus on some of the issues specific to protein-based complex tests. Technological points pertaining to mass spectrometry platforms and assessment of potential concerns important for assurance of safety and effectiveness of this type of assays when introduced into clinical diagnostic use, as well as general approaches for evaluating the performance of these devices, are discussed.

The journey to regulation of protein-based multiplex quantitative assays.

BACKGROUND: Clinical proteomics presents great promise in biology and medicine because of its potential for improving our understanding of diseases at the molecular level and for detecting disease-related biomarkers for diagnosis, prognosis, and prediction of therapeutic responses. To realize its full potential to improve clinical outcome for patients, proteomic studies have to be well designed, from biosample cohorts to data and statistical analyses. One key component in the biomarker development pipeline is the understanding of the regulatory science that evaluates diagnostic assay performance through rigorous analytical and clinical review criteria. CONTENT: The National Cancer Institute's Clinical Proteomic Technologies for Cancer (CPTC) initiative has proposed an intermediate preclinical "verification" step to close the gap between protein-based biomarker discovery and clinical qualification. In collaboration with the US Food and Drug Administration (FDA), the CPTC network investigators recently published 2 mock submission review documents, first-of-their-kind educational materials that may help the scientific community interested in developing products for the clinic in understanding the likely analytical evaluation requirements for multiplex protein technology-based diagnostic tests. CONCLUSIONS: Building on this momentum, the CPTC continues with this report its collaboration with the FDA, as well as its interactions with the AACC and the Centers for Medicare and Medicaid Services, to further the understanding of regulatory requirements for approving multiplex proteomic platform-based tests and analytically validating multiple analytes.

US FDA and personalized medicine: in vitro diagnostic regulatory perspective.

Personalized medicine has captured the attention of the public, including patients, healthcare providers, scientists, medical product manufacturers and many others. The US FDA will evaluate many of the products that will allow personalized medicine to be successfully implemented in the USA. This article addresses the FDA's approach to regulation of one component of personalized medicine, in vitro diagnostic devices. It also describes the FDA's efforts to integrate the various medical product regulatory authorities provided by Congress in the Federal Food, Drug and Cosmetic Act to develop effective mechanisms for oversight of medical products used to personalize treatment. Finally, it presents some of the current challenges in in vitro diagnostics oversight that may be of interest for personalized medicine.

Biomarkers to improve the benefit/risk balance for approved therapeutics: a US FDA perspective on personalized medicine.

This article highlights a current US FDA perspective concerning the use of biomarker-based diagnostics for personalized medicine. Specifically, current biomarkers that have application for improving the benefit/risk profile of already approved drugs are discussed. The success of biomarkers for use in personalized medicine depends on many factors, including proper evaluation of the usefulness of the biomarker for assessing the event of interest, and the safety and effectiveness of the diagnostic device used to measure the biomarker, which includes appropriate analytical and clinical validation. These points along with the many regulatory concerns regarding co-labeling of drugs and devices and future aspects, such as co-development, will be discussed in this regulatory science focus.

Protein-based multiplex assays: mock presubmissions to the US Food and Drug Administration.

As a part of ongoing efforts of the NCI-FDA Interagency Oncology Task Force subcommittee on molecular diagnostics, members of the Clinical Proteomic Technology Assessment for Cancer program of the National Cancer Institute have submitted 2 protein-based multiplex assay descriptions to the Office of In Vitro Diagnostic Device Evaluation and Safety, US Food and Drug Administration. The objective was to evaluate the analytical measurement criteria and studies needed to validate protein-based multiplex assays. Each submission described a different protein-based platform: a multiplex immunoaffinity mass spectrometry platform for protein quantification, and an immunological array platform quantifying glycoprotein isoforms. Submissions provided a mutually beneficial way for members of the proteomics and regulatory communities to identify the analytical issues that the field should address when developing protein-based multiplex clinical assays.

Analytical validation of protein-based multiplex assays: a workshop report by the NCI-FDA interagency oncology task force on molecular diagnostics.

Clinical proteomics has the potential to enable the early detection of cancer through the development of multiplex assays that can inform clinical decisions. However, there has been some uncertainty among translational researchers and developers as to the specific analytical measurement criteria needed to validate protein-based multiplex assays. To begin to address the causes of this uncertainty, a day-long workshop titled "Interagency Oncology Task Force Molecular Diagnostics Workshop" was held in which members of the proteomics and regulatory communities discussed many of the analytical evaluation issues that the field should address in development of protein-based multiplex assays for clinical use. This meeting report explores the issues raised at the workshop and details the recommendations that came out of the day's discussions, such as a workshop summary discussing the analytical evaluation issues that specific proteomic technologies should address when seeking US Food and Drug Administration approval.

Biomarkers for pharmacogenetic and pharmacogenomic studies: Locking down analytical performance.

The use of in vitro tests to detect and measure biomarkers provides promising avenues for development of new and better drugs, and will be central to the realization of personalized medicine. The importance of proper biomarker test assessment cannot be overemphasized. Whether the test is being used as part of drug development or ultimately used as a companion diagnostic, if the test result is to be meaningful, the test analytical performance must be well characterized. This article will outline important analytical validation issues to consider when developing and assessing an in vitro diagnostic test system for use in pharmacogenetic and pharmacogenomic studies.:

Biomarkers for pharmacogenetic and pharmacogenomic studies: special issues in analytical performance.

The use of in vitro tests to detect and measure biomarkers will be central to the realization of personalized medicine. The importance of proper biomarker test development processes cannot be overemphasized; whether the test is being used as part of drug development or ultimately for use as a companion diagnostic, if the test result is to be meaningful, the test analytical performance must be well characterized and reliable. This article will outline important design issues and special analytical validation issues to consider when developing and assessing an in vitro diagnostic test system for use in pharmacogenetic and pharmacogenomic studies. Of particular importance is understanding that good analytical performance is a result of a well-coordinated analytical system specifically designed to provide quality results.: