Evaluating eligibility criteria of oncology trials using real-world data and AI.

There is a growing focus on making clinical trials more inclusive but the design of trial eligibility criteria remains challenging1-3. Here we systematically evaluate the effect of different eligibility criteria on cancer trial populations and outcomes with real-world data using the computational framework of Trial Pathfinder. We apply Trial Pathfinder to emulate completed trials of advanced non-small-cell lung cancer using data from a nationwide database of electronic health records comprising 61,094 patients with advanced non-small-cell lung cancer. Our analyses reveal that many common criteria, including exclusions based on several laboratory values, had a minimal effect on the trial hazard ratios. When we used a data-driven approach to broaden restrictive criteria, the pool of eligible patients more than doubled on average and the hazard ratio of the overall survival decreased by an average of 0.05. This suggests that many patients who were not eligible under the original trial criteria could potentially benefit from the treatments. We further support our findings through analyses of other types of cancer and patient-safety data from diverse clinical trials. Our data-driven methodology for evaluating eligibility criteria can facilitate the design of more-inclusive trials while maintaining safeguards for patient safety.

Using Electronic Health Records to Derive Control Arms for Early Phase Single-Arm Lung Cancer Trials: Proof-of-Concept in Randomized Controlled Trials.

Oncology drug development increasingly relies on single-arm clinical trials. External controls (ECs) derived from electronic health record (EHR) databases may provide additional context. Patients from a US-based oncology EHR database were aligned with patients from randomized controlled trials (RCTs) and trial-specific eligibility criteria were applied to the EHR dataset. Overall survival (OS) in the EC-derived control arm was compared with OS in the RCT experimental arm. The primary outcome was OS, defined as time from randomization or treatment initiation (EHR) to death. Cox regression models were used to obtain effect estimates using EHR data. EC-derived hazard ratio estimates aligned closely with those from the corresponding RCT with one exception. Comparing log HRs among all RCT and EC results gave a Pearson correlation coefficient of 0.86. Properly selected control arms from contemporaneous EHR data could be used to put single-arm trials of OS in advanced non-small cell lung cancer into context.

An evaluation of the impact of missing deaths on overall survival analyses of advanced non-small cell lung cancer patients conducted in an electronic health records database.

PURPOSE: The aim of this study was to assess the impact of missing death data on survival analyses conducted in an oncology EHR-derived database. METHODS: The study was conducted using the Flatiron Health oncology database and the National Death Index (NDI) as a gold standard. Three analytic frameworks were evaluated in advanced non-small cell lung cancer (aNSCLC) patients: median overall survival [mOS]), relative risk estimates conducted within the EHR-derived database, and "external control arm" analyses comparing an experimental group augmented with mortality data from the gold standard to a control group from the EHR-derived database only. The hazard ratios (HRs) obtained within the EHR-derived database (91% sensitivity) and the external control arm analyses, were compared with results when both groups were augmented with mortality data from the gold standard. The above analyses were repeated using simulated lower mortality sensitivities to understand the impact of more extreme levels of missing deaths. RESULTS: Bias in mOS ranged from modest (0.6-0.9 mos.) in the EHR-derived cohort with (91% sensitivity) to substantial when lower sensitivities were generated through simulation (3.3-9.7 mos.). Overall, small differences were observed in the HRs for the EHR-derived cohort across comparative analyses when compared with HRs obtained using the gold standard data source. When only one treatment arm was subject to estimation bias, the bias was slightly more pronounced, but increased substantially when lower sensitivities were simulated. CONCLUSIONS: The impact on survival analysis is minimal with high mortality sensitivity with only modest impact associated within external control arm applications.

Psychometric validation of the electronic chronic pain questions (eCPQ) in a primary care setting.

OBJECTIVE: Collecting data that helps evaluate different types of pain may improve physicians' decision-making with regard to treatment selection and on-going monitoring of patients. To date, no chronic pain assessments have been widely implemented in primary care. The aim of this study was to psychometrically validate the electronic Chronic Pain Questions (eCPQ) in a primary care setting. RESEARCH DESIGN AND METHODS: All men and women ≥18 years arriving at two similar primary care clinics in southeastern Michigan were invited to participate. Clinic staff verbally administered the eCPQ to patients and recorded their answers into the electronic medical record (EMR) prior to physician consultation with results available for physician review. Concurrent validity was assessed using Spearman correlations between eCPQ and patient-completed ancillary measures. Known-group validity was assessed by stratifying patients on self-reported chronic pain as well as by pain diagnosis (i.e. ICD-9 codes). To compare patients with chronic pain versus no chronic pain t-tests and chi-square tests were performed. Reproducibility was assessed between interviewer- and self-administration over time. RESULTS: A total of 534 patients were invited to participate and 455 patients consented to take part in the study (85.2% response rate); 395 patients had analyzable eCPQ data; 70.1% were Caucasian; 68.1% female; mean age was 43.4; 52.7% (n = 208) self-reported chronic pain. Correlations between eCPQ and ancillary measures supported concurrent validity. Excellent discrimination between groups was evidenced based on self-reported chronic pain and ICD-9 diagnosis. Patients with self-reported chronic pain reported significantly (p < .0001) higher pain ratings and greater interference with usual activities, sleep, and mood than those without chronic pain. Test-retest reliability between modes (interviewer- vs. self-administration) was excellent as was reproducibility based on self-administration of the eCPQ at two separate time points. Key limitations: Discriminant validity was determined by comparing participants based on ICD codes. Utilizing ICD codes to identify individuals with chronic pain may not be a reliable approach as it is dependent upon providers accurately and consistently entering chronic pain diagnoses in the EMR. CONCLUSIONS: The eCPQ has sound psychometric measurement properties, including concurrent validity, discriminant validity, and reproducibility. The eCPQ appears to be useful to identify patients with chronic pain and to assess and monitor symptoms over time.