Analytical validation of the Percepta genomic sequencing classifier; an RNA next generation sequencing assay for the assessment of Lung Cancer risk of suspicious pulmonary nodules.

BACKGROUND: Bronchoscopy is a common procedure used for evaluation of suspicious lung nodules, but the low diagnostic sensitivity of bronchoscopy often results in inconclusive results and delays in treatment. Percepta Genomic Sequencing Classifier (GSC) was developed to assist with patient management in cases where bronchoscopy is inconclusive. Studies have shown that exposure to tobacco smoke alters gene expression in airway epithelial cells in a way that indicates an increased risk of developing lung cancer. Percepta GSC leverages this idea of a molecular "field of injury" from smoking and was developed using RNA sequencing data generated from lung bronchial brushings of the upper airway. A Percepta GSC score is calculated from an ensemble of machine learning algorithms utilizing clinical and genomic features and is used to refine a patient's risk stratification. METHODS: The objective of the analysis described and reported here is to validate the analytical performance of Percepta GSC. Analytical performance studies characterized the sensitivity of Percepta GSC test results to input RNA quantity, the potentially interfering agents of blood and genomic DNA, and the reproducibility of test results within and between processing runs and between laboratories. RESULTS: Varying the amount of input RNA into the assay across a nominal range had no significant impact on Percepta GSC classifier results. Bronchial brushing RNA contaminated with up to 10% genomic DNA by nucleic acid mass also showed no significant difference on classifier results. The addition of blood RNA, a potential contaminant in the bronchial brushing sample, caused no change to classifier results at up to 11% contamination by RNA proportion. Percepta GSC scores were reproducible between runs, within runs, and between laboratories, varying within less than 4% of the total score range (standard deviation of 0.169 for scores on 4.57 scale). CONCLUSIONS: The analytical sensitivity, analytical specificity, and reproducibility of Percepta GSC laboratory results were successfully demonstrated under conditions of expected day to day variation in testing. Percepta GSC test results are analytically robust and suitable for routine clinical use.

Utility of a Molecular Classifier as a Complement to High-Resolution Computed Tomography to Identify Usual Interstitial Pneumonia.

Rationale: Usual interstitial pneumonia (UIP) is the defining morphology of idiopathic pulmonary fibrosis (IPF). Guidelines for IPF diagnosis conditionally recommend surgical lung biopsy for histopathology diagnosis of UIP when radiology and clinical context are not definitive. A "molecular diagnosis of UIP" in transbronchial lung biopsy, the Envisia Genomic Classifier, accurately predicted histopathologic UIP.Objectives: We evaluated the combined accuracy of the Envisia Genomic Classifier and local radiology in the detection of UIP pattern.Methods: Ninety-six patients who had diagnostic lung pathology as well as a transbronchial lung biopsy for molecular testing with Envisia Genomic Classifier were included in this analysis. The classifier results were scored against reference pathology. UIP identified on high-resolution computed tomography (HRCT) as documented by features in local radiologists' reports was compared with histopathology.Measurements and Main Results: In 96 patients, the Envisia Classifier achieved a specificity of 92.1% (confidence interval [CI],78.6-98.3%) and a sensitivity of 60.3% (CI, 46.6-73.0%) for histology-proven UIP pattern. Local radiologists identified UIP in 18 of 53 patients with UIP histopathology, with a sensitivity of 34.0% (CI, 21.5-48.3%) and a specificity of 96.9% (CI, 83.8-100%). In conjunction with HRCT patterns of UIP, the Envisia Classifier results identified 24 additional patients with UIP (sensitivity 79.2%; specificity 90.6%).Conclusions: In 96 patients with suspected interstitial lung disease, the Envisia Genomic Classifier identified UIP regardless of HRCT pattern. These results suggest that recognition of a UIP pattern by the Envisia Genomic Classifier combined with HRCT and clinical factors in a multidisciplinary discussion may assist clinicians in making an interstitial lung disease (especially IPF) diagnosis without the need for a surgical lung biopsy.

Improving lung cancer risk stratification leveraging whole transcriptome RNA sequencing and machine learning across multiple cohorts.

BACKGROUND: Bronchoscopy for suspected lung cancer has low diagnostic sensitivity, rendering many inconclusive results. The Bronchial Genomic Classifier (BGC) was developed to help with patient management by identifying those with low risk of lung cancer when bronchoscopy is inconclusive. The BGC was trained and validated on patients in the Airway Epithelial Gene Expression in the Diagnosis of Lung Cancer (AEGIS) trials. A modern patient cohort, the BGC Registry, showed differences in key clinical factors from the AEGIS cohorts, with less smoking history, smaller nodules and older age. Additionally, we discovered interfering factors (inhaled medication and sample collection timing) that impacted gene expressions and potentially disguised genomic cancer signals. METHODS: In this study, we leveraged multiple cohorts and next generation sequencing technology to develop a robust Genomic Sequencing Classifier (GSC). To address demographic composition shift and interfering factors, we synergized three algorithmic strategies: 1) ensemble of clinical dominant and genomic dominant models; 2) development of hierarchical regression models where the main effects from clinical variables were regressed out prior to the genomic impact being fitted in the model; and 3) targeted placement of genomic and clinical interaction terms to stabilize the effect of interfering factors. The final GSC model uses 1232 genes and four clinical covariates - age, pack-years, inhaled medication use, and specimen collection timing. RESULTS: In the validation set (N = 412), the GSC down-classified low and intermediate pre-test risk subjects to very low and low post-test risk with a specificity of 45% (95% CI 37-53%) and a sensitivity of 91% (95%CI 81-97%), resulting in a negative predictive value of 95% (95% CI 89-98%). Twelve percent of intermediate pre-test risk subjects were up-classified to high post-test risk with a positive predictive value of 65% (95%CI 44-82%), and 27% of high pre-test risk subjects were up-classified to very high post-test risk with a positive predictive value of 91% (95% CI 78-97%). CONCLUSIONS: The GSC overcame the impact of interfering factors and achieved consistent performance across multiple cohorts. It demonstrated diagnostic accuracy in both down- and up-classification of cancer risk, providing physicians actionable information for many patients with inconclusive bronchoscopy.

Use of a molecular classifier to identify usual interstitial pneumonia in conventional transbronchial lung biopsy samples: a prospective validation study.

BACKGROUND: In the appropriate clinical setting, the diagnosis of idiopathic pulmonary fibrosis (IPF) requires a pattern of usual interstitial pneumonia to be present on high-resolution chest CT (HRCT) or surgical lung biopsy. A molecular usual interstitial pneumonia signature can be identified by a machine learning algorithm in less-invasive transbronchial lung biopsy samples. We report prospective findings for the clinical validity and utility of this molecular test. METHODS: We prospectively recruited 237 patients for this study from those enrolled in the Bronchial Sample Collection for a Novel Genomic Test (BRAVE) study in 29 US and European sites. Patients were undergoing evaluation for interstitial lung disease and had had samples obtained by clinically indicated surgical or transbronchial biopsy or cryobiopsy for pathology. Histopathological diagnoses were made by experienced pathologists. Available HRCT scans were reviewed centrally. Three to five transbronchial lung biopsy samples were collected from all patients specifically for this study, pooled by patient, and extracted for transcriptomic sequencing. After exclusions, diagnostic histopathology and RNA sequence data from 90 patients were used to train a machine learning algorithm (Envisia Genomic Classifier, Veracyte, San Francisco, CA, USA) to identify a usual interstitial pneumonia pattern. The primary study endpoint was validation of the classifier in 49 patients by comparison with diagnostic histopathology. To assess clinical utility, we compared the agreement and confidence level of diagnosis made by central multidisciplinary teams based on anonymised clinical information and radiology results plus either molecular classifier or histopathology results. FINDINGS: The classifier identified usual interstitial pneumonia in transbronchial lung biopsy samples from 49 patients with 88% specificity (95% CI 70-98) and 70% sensitivity (47-87). Among 42 of these patients who had possible or inconsistent usual interstitial pneumonia on HRCT, the classifier showed 81% positive predictive value (95% CI 54-96) for underlying biopsy-proven usual interstitial pneumonia. In the clinical utility analysis, we found 86% agreement (95% CI 78-92) between clinical diagnoses using classifier results and those using histopathology data. Diagnostic confidence was improved by the molecular classifier results compared with histopathology results in 18 with IPF diagnoses (proportion of diagnoses that were confident or provisional with high confidence 89% vs 56%, p=0·0339) and in all 48 patients with non-diagnostic pathology or non-classifiable fibrosis histopathology (63% vs 42%, p=0·0412). INTERPRETATION: The molecular test provided an objective method to aid clinicians and multidisciplinary teams in ascertaining a diagnosis of IPF, particularly for patients without a clear radiological diagnosis, in samples that can be obtained by a less invasive method. Further prospective clinical validation and utility studies are planned. FUNDING: Veracyte.

Identification of usual interstitial pneumonia pattern using RNA-Seq and machine learning: challenges and solutions.

BACKGROUND: We developed a classifier using RNA sequencing data that identifies the usual interstitial pneumonia (UIP) pattern for the diagnosis of idiopathic pulmonary fibrosis. We addressed significant challenges, including limited sample size, biological and technical sample heterogeneity, and reagent and assay batch effects. RESULTS: We identified inter- and intra-patient heterogeneity, particularly within the non-UIP group. The models classified UIP on transbronchial biopsy samples with a receiver-operating characteristic area under the curve of ~ 0.9 in cross-validation. Using in silico mixed samples in training, we prospectively defined a decision boundary to optimize specificity at ≥85%. The penalized logistic regression model showed greater reproducibility across technical replicates and was chosen as the final model. The final model showed sensitivity of 70% and specificity of 88% in the test set. CONCLUSIONS: We demonstrated that the suggested methodologies appropriately addressed challenges of the sample size, disease heterogeneity and technical batch effects and developed a highly accurate and robust classifier leveraging RNA sequencing for the classification of UIP.

Analytical performance of Envisia: a genomic classifier for usual interstitial pneumonia.

BACKGROUND: Clinical guidelines specify that diagnosis of interstitial pulmonary fibrosis (IPF) requires identification of usual interstitial pneumonia (UIP) pattern. While UIP can be identified by high resolution CT of the chest, the results are often inconclusive, making surgical lung biopsy necessary to reach a definitive diagnosis (Raghu et al., Am J Respir Crit Care Med 183(6):788-824, 2011). The Envisia genomic classifier differentiates UIP from non-UIP pathology in transbronchial biopsies (TBB), potentially allowing patients to avoid an invasive procedure (Brown et al., Am J Respir Crit Care Med 195:A6792, 2017). To ensure patient safety and efficacy, a laboratory developed test (LDT) must meet strict regulatory requirements for accuracy, reproducibility and robustness. The analytical characteristics of the Envisia test are assessed and reported here. METHODS: The Envisia test utilizes total RNA extracted from TBB samples to perform Next Generation RNA Sequencing. The gene count data from 190 genes are then input to the Envisia genomic classifier, a machine learning algorithm, to output either a UIP or non-UIP classification result. We characterized the stability of RNA in TBBs during collection and shipment, and evaluated input RNA mass and proportions on the limit of detection of UIP. We evaluated potentially interfering substances such as blood and genomic DNA. Intra-run, inter-run, and inter-laboratory reproducibility of test results were also characterized. RESULTS: RNA content within TBBs preserved in RNAprotect is stable for up to 14 days with no detectable change in RNA quality. The Envisia test is tolerant to variation in RNA input (5 to 30 ng), with no impact on classifier results. The Envisia test can tolerate dilution of non-UIP and UIP classification signals at the RNA level by up to 60% and 20%, respectively. Analytical specificity studies utilizing UIP and non-UIP samples mixed with genomic DNA (up to 30% relative input) demonstrated no impact to classifier results. The Envisia test tolerates up to 22% of blood contamination, well beyond the level observed in TBBs. The test is reproducible from RNA extraction through to Envisia test result (standard deviation of 0.20 for Envisia classification scores on > 7-unit scale). CONCLUSIONS: The Envisia test demonstrates the robust analytical performance required of an LDT. Envisia can be used to inform the diagnoses of patients with suspected IPF.

Usual Interstitial Pneumonia Can Be Detected in Transbronchial Biopsies Using Machine Learning.

RATIONALE: Usual interstitial pneumonia (UIP) is the histopathologic hallmark of idiopathic pulmonary fibrosis. Although UIP can be detected by high-resolution computed tomography of the chest, the results are frequently inconclusive, and pathology from transbronchial biopsy (TBB) has poor sensitivity. Surgical lung biopsy may be necessary for a definitive diagnosis. OBJECTIVES: To develop a genomic classifier in tissue obtained by TBB that distinguishes UIP from non-UIP, trained against central pathology as the reference standard. METHODS: Exome enriched RNA sequencing was performed on 283 TBBs from 84 subjects. Machine learning was used to train an algorithm with high rule-in (specificity) performance using specimens from 53 subjects. Performance was evaluated by cross-validation and on an independent test set of specimens from 31 subjects. We explored the feasibility of a single molecular test per subject by combining multiple TBBs from upper and lower lobes. To address whether classifier accuracy depends upon adequate alveolar sampling, we tested for correlation between classifier accuracy and expression of alveolar-specific genes. RESULTS: The top-performing algorithm distinguishes UIP from non-UIP conditions in single TBB samples with an area under the receiver operator characteristic curve (AUC) of 0.86, with specificity of 86% (confidence interval = 71-95%) and sensitivity of 63% (confidence interval = 51-74%) (31 test subjects). Performance improves to an AUC of 0.92 when three to five TBB samples per subject are combined at the RNA level for testing. Although we observed a wide range of type I and II alveolar-specific gene expression in TBBs, expression of these transcripts did not correlate with classifier accuracy. CONCLUSIONS: We demonstrate proof of principle that genomic analysis and machine learning improves the utility of TBB for the diagnosis of UIP, with greater sensitivity and specificity than pathology in TBB alone. Combining multiple individual subject samples results in increased test accuracy over single sample testing. This approach requires validation in an independent cohort of subjects before application in the clinic.

Analytical Performance of a Gene Expression Classifier for Medullary Thyroid Carcinoma.

BACKGROUND: The aim of this study was to demonstrate the analytical validity of an RNA classifier for medullary thyroid carcinoma (MTC). METHODS: Fresh-frozen tissue specimens were obtained from commercial sources, and MTC diagnoses were confirmed by histopathology review. De-identified patient fine-needle aspiration biopsies (FNABs) and whole blood from normal donors were obtained. Total RNA was extracted, amplified, and hybridized to custom microarrays for gene expression analysis. Gene expression data were normalized and classified via a machine learning algorithm. Positive control materials were produced from MTC tissues and tested across multiple experiments and laboratories. Twenty-seven MTC tissue specimens were used to evaluate the sensitivity of the MTC classifier. Gene expression data from tissues and FNABs were used to model classifier response to mixtures of MTC samples with normal thyroid tissue, a benign thyroid nodule, a Hürthle cell adenoma, and whole blood. Select mixture conditions were confirmed in vitro. Assay tolerance to RNA input variation (5-25 ng) and genomic DNA contamination (30% by mass) was evaluated. The intra- and inter-run reproducibility and inter-laboratory accuracy of MTC classifier results were characterized. RESULTS: The MTC classifier sensitivity of 96.3% [confidence interval 81.0-99.9%] was determined retrospectively using 27 MTC confirmed tissue specimens. One false-negative result in a necrotic tissue implicated sample necrosis in reduced classifier sensitivity. Dilution modeling of MTC samples with normal or benign tissues showed consistent detection of MTC down to 20% sample proportions, with in vitro confirmation of 20% analytical sensitivity. Classifier tolerance to RNA input variation (5-25 ng), genomic DNA contamination (30% by mass), and an interfering substance (blood) was demonstrated with 100% accurate classifier results under all tested conditions. The maximum observed run-to-run score difference for a single FNAB sample was ∼1 unit compared with the average score difference between 38 MTC and non-MTC FNABs of ∼32 units. MTC classifier results for 20 tissues processed from total RNA in two different laboratories showed 100% concordance. CONCLUSIONS: The MTC classifier, offered as part of the routine molecular testing of cytology-indeterminate thyroid nodules, demonstrates robust analytical sensitivity, specificity, accuracy, and reproducibility.

Analytical performance verification of a molecular diagnostic for cytology-indeterminate thyroid nodules.

OBJECTIVE: Our objective was to verify the analytical performance of the Afirma gene expression classifier (GEC) in the classification of cytologically indeterminate thyroid nodule fine-needle aspirates (FNAs). DESIGN: Analytical performance studies were designed to characterize the stability of RNA in FNAs during collection and shipment, analytical sensitivity as applied to input RNA concentration and malignant/benign FNA mixtures, analytical specificity (i.e. potentially interfering substances) as tested on blood and genomic DNA, and assay performance studies including intra-nodule, intraassay, inter-assay, and inter-laboratory reproducibility. RESULTS: RNA content within FNAs preserved in FNAProtect is stable for up to 6 d at room temperature with no changes in RNA yield (P = 0.58) or quality (P = 0.56). FNA storage and shipping temperatures were found to have no significant effect on GEC scores (P = 0.55) or calls (100% concordance). Analytical sensitivity studies demonstrated tolerance to variation in RNA input (5-25 ng) and to the dilution of malignant FNA material down to 20%. Analytical specificity studies using malignant samples mixed with blood (up to 83%) and genomic DNA (up to 30%) demonstrated negligible assay interference with respect to false-negative calls, although benign FNA samples mixed with relatively high proportions of blood demonstrated a potential for false-positive calls. The test is reproducible from extraction through GEC result, including variation across operators, runs, reagent lots, and laboratories (sd of 0.158 for scores on a >6 unit scale). CONCLUSIONS: Analytical sensitivity, analytical specificity, robustness, and quality control of the GEC were successfully verified, indicating its suitability for clinical use.

Aberrant recombination involving the granzyme locus occurs in Atm-/- T-cell lymphomas.

Ataxia telangiectasia (A-T) is an autosomal recessive disease caused by loss of function of the serine/threonine protein kinase ATM (ataxia telangiectasia mutated). A-T patients have a 250-700-fold increased risk of developing lymphomas and leukemias which are typically highly invasive and proliferative. In addition, a subset of adult acute lymphoblastic leukemias and aggressive B-cell chronic lymphocytic leukemias that occur in the general population show loss of heterozygosity for ATM. To define the specific role of ATM in lymphomagenesis, we studied T-cell lymphomas isolated from mice with mutations in ATM and/or p53 using cytogenetic analysis and mRNA transcriptional profiling. The analyses identified genes misregulated as a consequence of the amplifications, deletions and translocation events arising as a result of ATM loss. A specific recurrent disruption of the granzyme gene family locus was identified resulting in an aberrant granzyme B/C fusion product. The combined application of cytogenetic and gene expression approaches identified specific loci and genes that define the pathway of initiation and progression of lymphoreticular malignancies in the absence of ATM.

Meiotic S-phase damage activates recombination without checkpoint arrest.

Checkpoints operate during meiosis to ensure the completion of DNA synthesis and programmed recombination before the initiation of meiotic divisions. Studies in the fission yeast Schizosaccharomyces pombe suggest that the meiotic response to DNA damage due to a failed replication checkpoint response differs substantially from the vegetative response, and may be influenced by the presence of homologous chromosomes. The checkpoint responses to DNA damage during fission yeast meiosis are not well characterized. Here we report that DNA damage induced during meiotic S-phase does not activate checkpoint arrest. We also find that in wild-type cells, markers for DNA breaks can persist at least to the first meiotic division. We also observe increased spontaneous S-phase damage in checkpoint mutants, which is repaired by recombination without activating checkpoint arrest. Our results suggest that fission yeast meiosis is exceptionally tolerant of DNA damage, and that some forms of spontaneous S-phase damage can be repaired by recombination without activating checkpoint arrest.

Software and methods for oligonucleotide and cDNA array data analysis.

Two HTML-based programs were developed to analyze and filter gene-expression data: 'Bullfrog' for Affymetrix oligonucleotide arrays and 'Spot' for custom cDNA arrays. The programs provide intuitive data-filtering tools through an easy-to-use interface. A background subtraction and normalization program for cDNA arrays was also built that provides an informative summary report with data-quality assessments. These programs are freeware to aid in the analysis of gene-expression results and facilitate the search for genes responsible for interesting biological processes and phenotypes.