SEETrials: Leveraging Large Language Models for Safety and Efficacy Extraction in Oncology Clinical Trials.

Background: Initial insights into oncology clinical trial outcomes are often gleaned manually from conference abstracts. We aimed to develop an automated system to extract safety and efficacy information from study abstracts with high precision and fine granularity, transforming them into computable data for timely clinical decision-making. Methods: We collected clinical trial abstracts from key conferences and PubMed (2012-2023). The SEETrials system was developed with four modules: preprocessing, prompt modeling, knowledge ingestion and postprocessing. We evaluated the system's performance qualitatively and quantitatively and assessed its generalizability across different cancer types- multiple myeloma (MM), breast, lung, lymphoma, and leukemia. Furthermore, the efficacy and safety of innovative therapies, including CAR-T, bispecific antibodies, and antibody-drug conjugates (ADC), in MM were analyzed across a large scale of clinical trial studies. Results: SEETrials achieved high precision (0.958), recall (sensitivity) (0.944), and F1 score (0.951) across 70 data elements present in the MM trial studies Generalizability tests on four additional cancers yielded precision, recall, and F1 scores within the 0.966-0.986 range. Variation in the distribution of safety and efficacy-related entities was observed across diverse therapies, with certain adverse events more common in specific treatments. Comparative performance analysis using overall response rate (ORR) and complete response (CR) highlighted differences among therapies: CAR-T (ORR: 88%, 95% CI: 84-92%; CR: 95%, 95% CI: 53-66%), bispecific antibodies (ORR: 64%, 95% CI: 55-73%; CR: 27%, 95% CI: 16-37%), and ADC (ORR: 51%, 95% CI: 37-65%; CR: 26%, 95% CI: 1-51%). Notable study heterogeneity was identified (>75% I 2 heterogeneity index scores) across several outcome entities analyzed within therapy subgroups. Conclusion: SEETrials demonstrated highly accurate data extraction and versatility across different therapeutics and various cancer domains. Its automated processing of large datasets facilitates nuanced data comparisons, promoting the swift and effective dissemination of clinical insights.

AutoCriteria: a generalizable clinical trial eligibility criteria extraction system powered by large language models.

OBJECTIVES: We aim to build a generalizable information extraction system leveraging large language models to extract granular eligibility criteria information for diverse diseases from free text clinical trial protocol documents. We investigate the model's capability to extract criteria entities along with contextual attributes including values, temporality, and modifiers and present the strengths and limitations of this system. MATERIALS AND METHODS: The clinical trial data were acquired from https://ClinicalTrials.gov/. We developed a system, AutoCriteria, which comprises the following modules: preprocessing, knowledge ingestion, prompt modeling based on GPT, postprocessing, and interim evaluation. The final system evaluation was performed, both quantitatively and qualitatively, on 180 manually annotated trials encompassing 9 diseases. RESULTS: AutoCriteria achieves an overall F1 score of 89.42 across all 9 diseases in extracting the criteria entities, with the highest being 95.44 for nonalcoholic steatohepatitis and the lowest of 84.10 for breast cancer. Its overall accuracy is 78.95% in identifying all contextual information across all diseases. Our thematic analysis indicated accurate logic interpretation of criteria as one of the strengths and overlooking/neglecting the main criteria as one of the weaknesses of AutoCriteria. DISCUSSION: AutoCriteria demonstrates strong potential to extract granular eligibility criteria information from trial documents without requiring manual annotations. The prompts developed for AutoCriteria generalize well across different disease areas. Our evaluation suggests that the system handles complex scenarios including multiple arm conditions and logics. CONCLUSION: AutoCriteria currently encompasses a diverse range of diseases and holds potential to extend to more in the future. This signifies a generalizable and scalable solution, poised to address the complexities of clinical trial application in real-world settings.

Analysis of real-world data to investigate evolving treatment sequencing patterns in advanced non-small cell lung cancers and their impact on survival.

Background: Although optimal sequencing of systemic therapy in cancer care is critical to achieving maximal clinical benefit, there is a lack of analysis of treatment sequencing in advanced non-small cell lung cancer (aNSCLC) in real-world settings. Methods: A retrospective cohort study of 13,340 lung cancer patients within the Mount Sinai Health System (MSHS) was performed. Systemic therapy data of aNSCLC in 2,106 patients was the starting point in our analysis to investigate how treatment sequencing has evolved, the impact of sequencing patterns on clinical outcomes, and the effectiveness of 2nd line chemotherapy after patients progressed on immune checkpoint inhibitor (ICI)-based therapy as the 1st line of therapy (LOT). Results: There is a significant shift to more ICI-based therapy and multiple lines of targeted therapy after 2015. We compared clinical outcomes of two patient populations with different treatment sequencing patterns, with the 1st group receiving chemotherapy as the 1st LOT followed by ICI-based treatment, and the 2nd group treated in the opposite order receiving a 1st line ICI-containing regimen followed by a 2nd line chemotherapy. No statistically significant difference in overall survival (OS) was observed between the two groups [group 2 vs. group 1, adjusted hazard ratio (aHR) =1.36, P=0.39]. We assessed the efficacy of the 2nd line chemotherapy in three patient populations given either 1st line ICI single agent, 1st line ICI-chemotherapy combination, or 1st line chemotherapy alone, there was no statistically significant difference in time-to-next treatment (TTNT) and in OS among the three patient groups. Conclusions: Analysis of real-world data has shown two treatment sequencing patterns in aNSCLC, ICI followed by chemotherapy or chemotherapy followed by ICI, achieved similar clinical benefit. The chemotherapies routinely used following platinum doublet 1st LOT, is effective as the 2nd line option after ICI-chemotherapy combination in the 1st line setting.

Detecting Ground Glass Opacity Features in Patients With Lung Cancer: Automated Extraction and Longitudinal Analysis via Deep Learning-Based Natural Language Processing.

BACKGROUND: Ground-glass opacities (GGOs) appearing in computed tomography (CT) scans may indicate potential lung malignancy. Proper management of GGOs based on their features can prevent the development of lung cancer. Electronic health records are rich sources of information on GGO nodules and their granular features, but most of the valuable information is embedded in unstructured clinical notes. OBJECTIVE: We aimed to develop, test, and validate a deep learning-based natural language processing (NLP) tool that automatically extracts GGO features to inform the longitudinal trajectory of GGO status from large-scale radiology notes. METHODS: We developed a bidirectional long short-term memory with a conditional random field-based deep-learning NLP pipeline to extract GGO and granular features of GGO retrospectively from radiology notes of 13,216 lung cancer patients. We evaluated the pipeline with quality assessments and analyzed cohort characterization of the distribution of nodule features longitudinally to assess changes in size and solidity over time. RESULTS: Our NLP pipeline built on the GGO ontology we developed achieved between 95% and 100% precision, 89% and 100% recall, and 92% and 100% F1-scores on different GGO features. We deployed this GGO NLP model to extract and structure comprehensive characteristics of GGOs from 29,496 radiology notes of 4521 lung cancer patients. Longitudinal analysis revealed that size increased in 16.8% (240/1424) of patients, decreased in 14.6% (208/1424), and remained unchanged in 68.5% (976/1424) in their last note compared to the first note. Among 1127 patients who had longitudinal radiology notes of GGO status, 815 (72.3%) were reported to have stable status, and 259 (23%) had increased/progressed status in the subsequent notes. CONCLUSIONS: Our deep learning-based NLP pipeline can automatically extract granular GGO features at scale from electronic health records when this information is documented in radiology notes and help inform the natural history of GGO. This will open the way for a new paradigm in lung cancer prevention and early detection.

Role of EXO1 nuclease activity in genome maintenance, the immune response and tumor suppression in Exo1D173A mice.

DNA damage response pathways rely extensively on nuclease activity to process DNA intermediates. Exonuclease 1 (EXO1) is a pleiotropic evolutionary conserved DNA exonuclease involved in various DNA repair pathways, replication, antibody diversification, and meiosis. But, whether EXO1 facilitates these DNA metabolic processes through its enzymatic or scaffolding functions remains unclear. Here, we dissect the contribution of EXO1 enzymatic versus scaffolding activity by comparing Exo1DA/DA mice expressing a proven nuclease-dead mutant form of EXO1 to entirely EXO1-deficient Exo1-/- and EXO1 wild type Exo1+/+ mice. We show that Exo1DA/DA and Exo1-/- mice are compromised in canonical DNA repair processing, suggesting that the EXO1 enzymatic role is important for error-free DNA mismatch and double-strand break repair pathways. However, in non-canonical repair pathways, EXO1 appears to have a more nuanced function. Next-generation sequencing of heavy chain V region in B cells showed the mutation spectra of Exo1DA/DA mice to be intermediate between Exo1+/+ and Exo1-/- mice, suggesting that both catalytic and scaffolding roles of EXO1 are important for somatic hypermutation. Similarly, while overall class switch recombination in Exo1DA/DA and Exo1-/- mice was comparably defective, switch junction analysis suggests that EXO1 might fulfill an additional scaffolding function downstream of class switching. In contrast to Exo1-/- mice that are infertile, meiosis progressed normally in Exo1DA/DA and Exo1+/+ cohorts, indicating that a structural but not the nuclease function of EXO1 is critical for meiosis. However, both Exo1DA/DA and Exo1-/- mice displayed similar mortality and cancer predisposition profiles. Taken together, these data demonstrate that EXO1 has both scaffolding and enzymatic functions in distinct DNA repair processes and suggest a more composite and intricate role for EXO1 in DNA metabolic processes and disease.

Analysis of Real-World Data to Investigate the Impact of Race and Ethnicity on Response to Programmed Cell Death-1 and Programmed Cell Death-Ligand 1 Inhibitors in Advanced Non-Small Cell Lung Cancers.

BACKGROUND: Racial disparities among clinical trial participants present a challenge to assess whether trial results can be generalized into patients representing diverse races and ethnicities. The objective of this study was to evaluate the impact of race and ethnicity on treatment response in patients with advanced non-small cell lung cancer (aNSCLC) treated with programmed cell death-1 (PD-1) or programmed cell death-ligand 1 (PD-L1) inhibitors through analysis of real-world data (RWD). MATERIALS AND METHODS: A retrospective cohort study of 11,138 patients with lung cancer treated at hospitals within the Mount Sinai Health System was performed. Patients with confirmed aNSCLC who received anti-PD-1/PD-L1 treatment were analyzed for clinical outcomes. Our cohort included 249 patients with aNSCLC who began nivolumab, pembrolizumab, or atezolizumab treatment between November 2014 and December 2018. Time-to-treatment discontinuation (TTD) and overall survival (OS) were the analyzed clinical endpoints. RESULTS: After a median follow-up of 14.8 months, median TTD was 7.8 months (95% confidence interval, 5.4-not estimable [NE]) in 75 African American patients versus 4.6 (2.4-7.2) in 110 White patients (hazard ratio [HR], 0.63). Median OS was not reached (18.4-NE) in African American patients versus 11.6 months (9.7-NE) in White patients (HR, 0.58). Multivariable Cox regression conducted with potential confounders confirmed longer TTD (adjusted HR, 0.65) and OS (adjusted HR, 0.60) in African American versus White patients. Similar real-world response rate (42.6% vs. 43.5%) and disease control rate (59.6% vs. 56.5%) were observed in the African American and White patient populations. Further investigation revealed the African American patient group had lower incidence (14.7%) of putative hyperprogressive diseases (HPD) upon anti-PD-1/PD-L1 treatment than the White patient group (24.5%). CONCLUSION: Analysis of RWD showed longer TTD and OS in African American patients with aNSCLC treated with anti-PD-1/PD-L1 inhibitors. Lower incidence of putative HPD is a possible reason for the favorable outcomes in this patient population. IMPLICATIONS FOR PRACTICE: There is a significant underrepresentation of minority patients in randomized clinical trials, and this study demonstrates that real-world data can be used to investigate the impact of race and ethnicity on treatment response. In retrospective analysis of patients with advanced non-small cell lung cancer treated with programmed cell death-1 or programmed cell death-ligand 1 inhibitors, African American patients had significantly longer time-to-treatment discontinuation and longer overall survival. Analysis of real-world data can yield clinical insights and establish a more complete picture of medical interventions in routine clinical practice.

A composite biomarker of neutrophil-lymphocyte ratio and hemoglobin level correlates with clinical response to PD-1 and PD-L1 inhibitors in advanced non-small cell lung cancers.

BACKGROUND: Immune checkpoint inhibitors (ICIs) have been incorporated into various clinical oncology guidelines for systemic treatment of advanced non-small cell lung cancers (aNSCLC). However, less than 50% (and 20%) of the patients responded to the therapy as a first (or second) line of therapy. PD-L1 immunohistochemistry (IHC) is an extensively studied biomarker of response to ICI, but results from this test have equivocal predictive power. In order to identify other biomarkers that support clinical decision-making around whether to treat with ICIs or not, we performed a retrospective study of patients with aNSCLC who underwent ICI-based therapy in the Mount Sinai Health System between 2014 and 2019. METHODS: We analyzed data from standard laboratory tests performed in patients as a part of the routine clinical workup during treatment, including complete blood counts (CBC) and a comprehensive metabolic panel (CMP), to correlate test results with clinical response and survival. RESULTS: Of 11,138 NSCLC patients identified, 249 had been treated with ICIs. We found associations between high neutrophil-to-lymphocyte ratio (NLR ≥ 5) and poor survival in ICI-treated NSCLC. We further observed that sustained high NLR after initiation of treatment had a more profound impact on survival than baseline NLR, regardless of PD-L1 status. Hazard ratios when comparing patients with NLR ≥ 5 vs. NLR < 5 are 1.7 (p = 0.02), 3.4 (p = 4.2 × 10- 8), and 3.9 (p = 1.4 × 10- 6) at baseline, 2-8 weeks, and 8-14 weeks after treatment start, respectively. Mild anemia, defined as hemoglobin (HGB) less than 12 g/dL was correlated with survival independently of NLR. Finally, we developed a composite NLR and HGB biomarker. Patients with pretreatment NLR ≥ 5 and HGB < 12 g/dL had a median overall survival (OS) of 8.0 months (95% CI 4.5-11.5) compared to the rest of the cohort with a median OS not reached (95% CI 15.9-NE, p = 1.8 × 10- 5), and a hazard ratio of 2.6 (95% CI 1.7-4.1, p = 3.5 × 10- 5). CONCLUSIONS: We developed a novel composite biomarker for ICI-based therapy in NSCLC based on routine CBC tests, which may provide meaningful clinical utility to guide treatment decision. The results suggest that treatment of anemia to elevate HGB before initiation of ICI therapy may improve patient outcomes or the use of alternative non-chemotherapy containing regimens.

EXO1 suppresses double-strand break induced homologous recombination between diverged sequences in mammalian cells.

DNA double-strand breaks (DSBs) can be repaired through several mechanisms, including homologous recombination (HR). While HR between identical sequences is robust in mammalian cells, HR between diverged sequences is suppressed by DNA mismatch-repair (MMR) components such as MSH2. Exonuclease I (EXO1) interacts with the MMR machinery and has been proposed to act downstream of the mismatch recognition proteins in mismatch correction. EXO1 has also been shown to participate in extensive DSB end resection, an initial step in the HR pathway. To assess the contribution of EXO1 to HR in mammalian cells, DSB-inducible reporters were introduced into Exo1-/- mouse embryonic stem cells, including a novel GFP reporter containing several silent polymorphisms to monitor HR between diverged sequences. Compared to HR between identical sequences which was not clearly affected, HR between diverged sequences was substantially increased in Exo1-/- cells although to a lesser extent than seen in Msh2-/- cells. Thus, like canonical MMR proteins, EXO1 can restrain aberrant HR events between diverged sequence elements in the genome.

A Novel Chemotherapeutic Agent to Treat Tumors with DNA Mismatch Repair Deficiencies.

Impairing the division of cancer cells with genotoxic small molecules has been a primary goal to develop chemotherapeutic agents. However, DNA mismatch repair (MMR)-deficient cancer cells are resistant to most conventional chemotherapeutic agents. Here we have identified baicalein as a small molecule that selectively kills MutSα-deficient cancer cells. Baicalein binds preferentially to mismatched DNA and induces a DNA damage response in a MMR-dependent manner. In MutSα-proficient cells, baicalein binds to MutSα to dissociate CHK2 from MutSα leading to S-phase arrest and cell survival. In contrast, continued replication in the presence of baicalein in MutSα-deficient cells results in a high number of DNA double-strand breaks and ultimately leads to apoptosis. Consistently, baicalein specifically shrinks MutSα-deficient xenograft tumors and inhibits the growth of AOM-DSS-induced colon tumors in colon-specific MSH2 knockout mice. Collectively, baicalein offers the potential of an improved treatment option for patients with tumors with a DNA MMR deficiency. Cancer Res; 76(14); 4183-91. ©2016 AACR.

MSH2 Dysregulation Is Triggered by Proinflammatory Cytokine Stimulation and Is Associated with Liver Cancer Development.

Inflammation predisposes to tumorigenesis in various organs by potentiating a susceptibility to genetic aberrations. The mechanism underlying the enhanced genetic instability through chronic inflammation, however, is not clear. Here, we demonstrated that TNFα stimulation induced transcriptional downregulation of MSH2, a member of the mismatch repair family, via NF-κB-dependent miR-21 expression in hepatocytes. Liver cancers developed in ALB-MSH2(-) (/) (-)AID(+), ALB-MSH2(-) (/) (-), and ALB-AID(+) mice, in which MSH2 is deficient and/or activation-induced cytidine deaminase (AICDA) is expressed in cells with albumin-producing hepatocytes. The mutation signatures in the tumors developed in these models, especially ALB-MSH2(-) (/) (-)AID(+) mice, closely resembled those of human hepatocellular carcinoma. Our findings demonstrated that inflammation-mediated dysregulation of MSH2 may be a mechanism of genetic alterations during hepatocarcinogenesis. Cancer Res; 76(15); 4383-93. ©2016 AACR.

Mouse models of DNA mismatch repair in cancer research.

Germline mutations in DNA mismatch repair (MMR) genes are the cause of hereditary non-polyposis colorectal cancer/Lynch syndrome (HNPCC/LS) one of the most common cancer predisposition syndromes, and defects in MMR are also prevalent in sporadic colorectal cancers. In the past, the generation and analysis of mouse lines with knockout mutations in all of the known MMR genes has provided insight into how loss of individual MMR genes affects genome stability and contributes to cancer susceptibility. These studies also revealed essential functions for some of the MMR genes in B cell maturation and fertility. In this review, we will provide a brief overview of the cancer predisposition phenotypes of recently developed mouse models with targeted mutations in MutS and MutL homologs (Msh and Mlh, respectively) and their utility as preclinical models. The focus will be on mouse lines with conditional MMR mutations that have allowed more accurate modeling of human cancer syndromes in mice and that together with new technologies in gene targeting, hold great promise for the analysis of MMR-deficient intestinal tumors and other cancers which will drive the development of preventive and therapeutic treatment strategies.

An Msh2 conditional knockout mouse for studying intestinal cancer and testing anticancer agents.

BACKGROUND & AIMS: Mutations in the DNA mismatch repair (MMR) gene MSH2 cause Lynch syndromes I and II and sporadic colorectal cancers. Msh2(null) mice predominantly develop lymphoma and do not accurately recapitulate the colorectal cancer phenotype. METHODS: We generated and examined mice with a conditional Msh2 disruption (Msh2(LoxP)), permitting tissue-specific gene inactivation. ECMsh2(LoxP/LoxP) mice carried an EIIa-Cre transgene, and VCMsh2(LoxP/LoxP) mice carried a Villin-Cre transgene. We combined the VCMsh2(LoxP) allele with either Msh2(Delta7null) (VCMsh2(LoxP/null)) or Msh2(G674D) mutations (VCMsh2(LoxP/G674D)) to create allelic phase mutants. These mice were given cisplatin or 5-fluorouracil/leucovorin and oxaliplatin (FOLFOX), and their tumors were measured by magnetic resonance imaging. RESULTS: Embryonic fibroblasts from ECMsh2(LoxP/LoxP) mice do not express MSH2 and are MMR deficient. Reverse transcription, polymerase chain reaction, and immunohistochemistry from VCMsh2(LoxP/LoxP) mice demonstrated specific loss of Msh2 messenger RNA and protein from epithelial cells of the intestinal tract. Microsatellite instability was observed in all VCMsh2 strains and limited to the intestinal mucosa. Resulting adenomas and adenocarcinomas had somatic truncation mutations to the adenomatous polyposis coli (Apc) gene. VCMsh2(LoxP/LoxP) mice did not develop lymphoma. Comparison of allelic phase tumors revealed significant differences in multiplicity and size. When treated with cisplatin or FOLFOX, tumor size was reduced in VCMsh2(LoxP/G674D) but not VCMsh2(LoxP/null) tumors. The apoptotic response to FOLFOX was partially sustained in the intestinal mucosa of VCMsh2(LoxP/G674D) animals. CONCLUSIONS: Msh2(LoxP/LoxP) mice in combination with appropriate Cre recombinase transgenes have excellent potential for preclinical modeling of Lynch syndrome, MMR-deficient tumors of other tissue types, and use in drug development.

Examination of Msh6- and Msh3-deficient mice in class switching reveals overlapping and distinct roles of MutS homologues in antibody diversification.

Somatic hypermutation and class switch recombination (CSR) contribute to the somatic diversification of antibodies. It has been shown that MutS homologue (Msh)6 (in conjunction with Msh2) but not Msh3 is involved in generating A/T base substitutions in somatic hypermutation. However, their roles in CSR have not yet been reported. Here we show that Msh6(-)(/)(-) mice have a decrease in CSR, whereas Msh3(-)(/)(-) mice do not. When switch regions were analyzed for mutations, deficiency in Msh6 was associated with an increase in transition mutations at G/C basepairs, mutations at RGYW/WRCY hotspots, and a small increase in the targeting of G/C bases. In addition, Msh6(-)(/)(-) mice exhibited an increase in the targeting of recombination sites to GAGCT/GGGGT consensus repeats and hotspots in Sgamma3 but not in Smicro. In contrast to Msh2(-)(/)(-) mice, deficiency in Msh6 surprisingly did not change the characteristics of Smicro-Sgamma3 switch junctions. However, Msh6(-)(/)(-) mice exhibited a change in the positioning of Smicro and Sgamma3 junctions. Although none of these changes were seen in Msh3(-)(/)(-) mice, they had a higher percentage of large inserts in their switch junctions. Together, our data suggest that MutS homologues Msh2, Msh3, and Msh6 play overlapping and distinct roles during antibody diversification processes.