ABSTRACT
We present a general framework for developing a machine learning (ML) tool that supports clinician assessment of patient risk using electronic health record-derived real-world data and apply the framework to a quality improvement use case in an oncology setting to identify patients at risk for a near-term (60 day) emergency department (ED) visit who could potentially be eligible for a home-based acute care program. Framework steps include defining clinical quality improvement goals, model development and validation, bias assessment, retrospective and prospective validation, and deployment in clinical workflow. In the retrospective analysis for the use case, 8% of patient encounters were associated with a high risk (pre-defined as predicted probability ≥20%) for a near-term ED visit by the patient. Positive predictive value (PPV) and negative predictive value (NPV) for future ED events was 26% and 91%, respectively. Odds ratio (OR) of ED visit (high- vs. low-risk) was 3.5 (95% CI: 3.4-3.5). The model appeared to be calibrated across racial, gender, and ethnic groups. In the prospective analysis, 10% of patients were classified as high risk, 76% of whom were confirmed by clinicians as eligible for home-based acute care. PPV and NPV for future ED events was 22% and 95%, respectively. OR of ED visit (high- vs. low-risk) was 5.4 (95% CI: 2.6-11.0). The proposed framework for an ML-based tool that supports clinician assessment of patient risk is a stepwise development approach; we successfully applied the framework to an ED visit risk prediction use case.
ABSTRACT
PURPOSE: The Oncology Care Model (OCM) is Medicare's first alternative payment model program for patients with cancer. As of October 2017, participating practices were required to report biomarker testing of patients with advanced non-small-cell lung cancer (aNSCLC). Our objective was to evaluate the effect of this OCM reporting requirement on quality of care. METHODS: We selected patients with aNSCLC receiving care in practices in a nationwide de-identified electronic health record-derived database. We used an adjusted difference-in-differences (DID) logistic regression model to compare changes in biomarker testing rates (EGFR, ROS1, and ALK) and receipt of biomarker-guided therapy between patients in OCM versus non-OCM practices, before and after OCM implementation. RESULTS: The analysis included 14,048 patients from 45 OCM practices (n = 8,151) and 105 non-OCM practices (n = 5,897). The overall unadjusted rates for biomarker testing and receipt of biomarker-guided therapy increased over the study period (2011-2018) in both OCM (55.5% v 71.6%; 89.8% v 94.6%, respectively) and non-OCM (55.2% v 69.7%; 90.1% v 95.2%, respectively) practices. In the adjusted DID model, the rates of biomarker testing (odds ratio [OR], 1.09 [95% CI, 0.88 to 1.34]; P = .45) and receipt of biomarker-guided therapy (OR, 0.87 [95% CI, 0.52 to 1.45]; P = .58) were similar between OCM and non-OCM practices. CONCLUSION: OCM biomarker documentation and reporting requirements did not appear to increase the proportions of patients with aNSCLC who underwent testing or who received biomarker-guided therapy in OCM versus non-OCM practices.
