Development and Validation of an Automated, Real-Time Predictive Model for Postpartum Hemorrhage.

OBJECTIVE: To develop and validate a predictive model for postpartum hemorrhage that can be deployed in clinical care using automated, real-time electronic health record (EHR) data and to compare performance of the model with a nationally published risk prediction tool. METHODS: A multivariable logistic regression model was developed from retrospective EHR data from 21,108 patients delivering at a quaternary medical center between January 1, 2018, and April 30, 2022. Deliveries were divided into derivation and validation sets based on an 80/20 split by date of delivery. Postpartum hemorrhage was defined as blood loss of 1,000 mL or more in addition to postpartum transfusion of 1 or more units of packed red blood cells. Model performance was evaluated by the area under the receiver operating characteristic curve (AUC) and was compared with a postpartum hemorrhage risk assessment tool published by the CMQCC (California Maternal Quality Care Collaborative). The model was then programmed into the EHR and again validated with prospectively collected data from 928 patients between November 7, 2023, and January 31, 2024. RESULTS: Postpartum hemorrhage occurred in 235 of 16,862 patients (1.4%) in the derivation cohort. The predictive model included 21 risk factors and demonstrated an AUC of 0.81 (95% CI, 0.79-0.84) and calibration slope of 1.0 (Brier score 0.013). During external temporal validation, the model maintained discrimination (AUC 0.80, 95% CI, 0.72-0.84) and calibration (calibration slope 0.95, Brier score 0.014). This was superior to the CMQCC tool (AUC 0.69 [95% CI, 0.67-0.70], P<.001). The model maintained performance in prospective, automated data collected with the predictive model in real time (AUC 0.82 [95% CI, 0.73-0.91]). CONCLUSION: We created and temporally validated a postpartum hemorrhage prediction model, demonstrated its superior performance over a commonly used risk prediction tool, successfully coded the model into the EHR, and prospectively validated the model using risk factor data collected in real time. Future work should evaluate external generalizability and effects on patient outcomes; to facilitate this work, we have included the model coefficients and examples of EHR integration in the article.

Implementable Prediction of Pressure Injuries in Hospitalized Adults: Model Development and Validation.

Background: Numerous pressure injury prediction models have been developed using electronic health record data, yet hospital-acquired pressure injuries (HAPIs) are increasing, which demonstrates the critical challenge of implementing these models in routine care. Objective: To help bridge the gap between development and implementation, we sought to create a model that was feasible, broadly applicable, dynamic, actionable, and rigorously validated and then compare its performance to usual care (ie, the Braden scale). Methods: We extracted electronic health record data from 197,991 adult hospital admissions with 51 candidate features. For risk prediction and feature selection, we used logistic regression with a least absolute shrinkage and selection operator (LASSO) approach. To compare the model with usual care, we used the area under the receiver operating curve (AUC), Brier score, slope, intercept, and integrated calibration index. The model was validated using a temporally staggered cohort. Results: A total of 5458 HAPIs were identified between January 2018 and July 2022. We determined 22 features were necessary to achieve a parsimonious and highly accurate model. The top 5 features included tracheostomy, edema, central line, first albumin measure, and age. Our model achieved higher discrimination than the Braden scale (AUC 0.897, 95% CI 0.893-0.901 vs AUC 0.798, 95% CI 0.791-0.803). Conclusions: We developed and validated an accurate prediction model for HAPIs that surpassed the standard-of-care risk assessment and fulfilled necessary elements for implementation. Future work includes a pragmatic randomized trial to assess whether our model improves patient outcomes.

Financial impact of integrated specialty pharmacy efforts to avoid oral anticancer medication waste.

BACKGROUND: The growing number of oral anticancer medications represents a significant portion of pharmacy spending and can be costly for patients. Patients taking oral anticancer medications may experience frequent treatment changes following necessary safety and effectiveness monitoring, often resulting in medication waste. Strategies to avoid medication waste could alleviate the financial burden of these costly therapies on the payer and the patient. OBJECTIVE: To evaluate the impact on waste and cost avoidance of reviewing the amount of medication patients have on hand and the presence of upcoming follow-up (ie, provider visit, laboratory testing, or imaging) before requesting a prescription refill renewal for patients taking oral anticancer medications through an integrated health system specialty pharmacy. METHODS: We performed a retrospective review of patients filling oral anticancer medications prescribed by a Vanderbilt University Medical Center provider and dispensed by Vanderbilt Specialty Pharmacy between January 1, 2020, and December 31, 2020. Specialty pharmacists received a system-generated refill renewal request for oral anticancer medications when the final prescription refill was dispensed, prompting the pharmacist to review the patient's medical record for continued therapy appropriateness and to request a new prescription. If the patient had a sufficient supply on hand to last until an upcoming follow-up (ie, provider visit, imaging, or laboratory assessment), the pharmacist postponed the renewal until after the scheduled follow-up. Patients were included in the analysis if the refill renewal request was postponed after review of the amount of medication on hand and the presence of an upcoming follow-up. Medication outcomes (ie, continued, dose changed, held, medication changed to a different oral anticancer medication, or discontinued) resulting from the follow-up were collected. Cost avoidance in US dollars was assigned based on the outcome of follow-up by calculating the price per unit times the number of units that would have been unused or in excess of what was needed if the medication had been dispensed before the scheduled follow-up. The average wholesale price minus 20% (AWP-20%) and wholesale acquisition cost (WAC) were used to report a range of costs avoided over 12 months. RESULTS: The total cost avoidance over 12 months associated with postponing refill renewal requests in a large academic health system with an integrated specialty pharmacy ranged from $549,187.03 using WAC pricing to $751,994.99 using AWP-20% pricing, with a median cost avoidance per fill of $366.04 (WAC) to $1,931.18 (AWP-20%). Refill renewal requests were postponed in 159 instances for 135 unique patients. After follow-up, medications were continued unchanged in only 2% of postponed renewals, 56% of follow-ups resulted in medication discontinuations, 32% in dose changes, 5% in medication changes, and 5% in medication holds. CONCLUSIONS: Integrated health system specialty pharmacist postponement of refill requests after review of the amount of medication on hand and upcoming follow-up proved effective in avoiding waste and unnecessary medication costs in patients treated with oral anticancer medications at a large academic health system.

Identifying antinuclear antibody positive individuals at risk for developing systemic autoimmune disease: development and validation of a real-time risk model.

Objective: Positive antinuclear antibodies (ANAs) cause diagnostic dilemmas for clinicians. Currently, no tools exist to help clinicians interpret the significance of a positive ANA in individuals without diagnosed autoimmune diseases. We developed and validated a risk model to predict risk of developing autoimmune disease in positive ANA individuals. Methods: Using a de-identified electronic health record (EHR), we randomly chart reviewed 2,000 positive ANA individuals to determine if a systemic autoimmune disease was diagnosed by a rheumatologist. A priori, we considered demographics, billing codes for autoimmune disease-related symptoms, and laboratory values as variables for the risk model. We performed logistic regression and machine learning models using training and validation samples. Results: We assembled training (n = 1030) and validation (n = 449) sets. Positive ANA individuals who were younger, female, had a higher titer ANA, higher platelet count, disease-specific autoantibodies, and more billing codes related to symptoms of autoimmune diseases were all more likely to develop autoimmune diseases. The most important variables included having a disease-specific autoantibody, number of billing codes for autoimmune disease-related symptoms, and platelet count. In the logistic regression model, AUC was 0.83 (95% CI 0.79-0.86) in the training set and 0.75 (95% CI 0.68-0.81) in the validation set. Conclusion: We developed and validated a risk model that predicts risk for developing systemic autoimmune diseases and can be deployed easily within the EHR. The model can risk stratify positive ANA individuals to ensure high-risk individuals receive urgent rheumatology referrals while reassuring low-risk individuals and reducing unnecessary referrals.

Assessment of the CLOT (children's likelihood of thrombosis) real-time risk prediction model of hospital-associated venous thromboembolism in children with congenital heart disease.

BACKGROUND: Children with congenital heart disease (CHD) are at high risk for hospital-associated venous thromboembolism (HA-VTE). The children's likelihood of thrombosis (CLOT) trial validated a real-time predictive model for HA-VTE using data extracted from the EHR for pediatric inpatients. We tested the hypothesis that addition of CHD specific data would improve model prediction in the CHD population. METHODS: Model performance in CHD patients from 2010 to 2022, was assessed using 3 iterations of the CLOT model: 1) the original CLOT model, 2) the original model refit using only data from the CHD cohort, and 3) the model updated with the addition of cardiopulmonary bypass time, STAT Mortality Category, height, and weight as covariates. The discrimination of the three models was quantified and compared using AUROC. RESULTS: Our CHD cohort included 1457 patient encounters (median 2.0 IQR [0.5-5.2] years-old). HA-VTE was present in 5% of our CHD cohort versus 1% in the general pediatric population. Several features from the original model were associated with thrombosis in the CHD cohort including younger age, thrombosis history, infectious disease consultation, and EHR coding of a central venous line. Lower height and weight were associated with thrombosis. HA-VTE rate was 12% (18/149) amongst those with STAT Category 4-5 operation versus 4% (49/1256) with STAT Category 1-3 operation (P < .001). Longer cardiopulmonary bypass time (124 [92-205] vs. 94 [65-136] minutes, P < .001) was associated with thrombosis. The AUROC for the original (0.80 95% CI [0.75-0.85]), refit (0.85 [0.81-0.89]), and updated (0.86 [0.81-0.90]) models demonstrated excellent discriminatory ability within the CHD cohort. CONCLUSION: The automated approach with EHR data extraction makes the applicability of such models appealing for ease of clinical use. The addition of cardiac specific features improved model discrimination; however, this benefit was marginal compared to refitting the original model to the CHD cohort. This suggests strong predictive generalized models, such as CLOT, can be optimized for cohort subsets without additional data extraction, thus reducing cost of model development and deployment.

External validation of a predictive model for reintubation after cardiac surgery: A retrospective, observational study.

STUDY OBJECTIVE: Explore validation of a model to predict patients' risk of failing extubation, to help providers make informed, data-driven decisions regarding the optimal timing of extubation. DESIGN: We performed temporal, geographic, and domain validations of a model for the risk of reintubation after cardiac surgery by assessing its performance on data sets from three academic medical centers, with temporal validation using data from the institution where the model was developed. SETTING: Three academic medical centers in the United States. PATIENTS: Adult patients arriving in the cardiac intensive care unit with an endotracheal tube in place after cardiac surgery. INTERVENTIONS: Receiver operating characteristic (ROC) curves and concordance statistics were used as measures of discriminative ability, and calibration curves and Brier scores were used to assess the model's predictive ability. MEASUREMENTS: Temporal validation was performed in 1642 patients with a reintubation rate of 4.8%, with the model demonstrating strong discrimination (optimism-corrected c-statistic 0.77) and low predictive error (Brier score 0.044) but poor model precision and recall (Optimal F1 score 0.29). Combined domain and geographic validation were performed in 2041 patients with a reintubation rate of 1.5%. The model displayed solid discriminative ability (optimism-corrected c-statistic = 0.73) and low predictive error (Brier score = 0.0149) but low precision and recall (Optimal F1 score = 0.13). Geographic validation was performed in 2489 patients with a reintubation rate of 1.6%, with the model displaying good discrimination (optimism-corrected c-statistic = 0.71) and predictive error (Brier score = 0.0152) but poor precision and recall (Optimal F1 score = 0.13). MAIN RESULTS: The reintubation model displayed strong discriminative ability and low predictive error within each validation cohort. CONCLUSIONS: Future work is needed to explore how to optimize models before local implementation.

Model-Guided Decision-Making for Thromboprophylaxis and Hospital-Acquired Thromboembolic Events Among Hospitalized Children and Adolescents: The CLOT Randomized Clinical Trial.

Importance: Rates of hospital-acquired venous thromboembolism (HA-VTE) are increasing among pediatric patients. Identifying at-risk patients for whom prophylactic interventions should be considered remains challenging. Objective: To determine whether use of a previously validated HA-VTE prognostic model, together with pediatric hematologist review, could reduce pediatric inpatient rates of HA-VTE. Design, Setting, and Participants: This pragmatic randomized clinical trial was performed from November 2, 2020, through January 31, 2022, at a single-center academic children's hospital (Monroe Carell Jr Children's Hospital at Vanderbilt). All pediatric hospital admissions (aged <22 years) under inpatient status were included and randomized. Intervention: All patients had an HA-VTE probability automatically calculated daily, which was visible to the hematology research team for patients in the intervention group. Patients with an elevated risk (predicted probability ≥2.5%) underwent additional medical record review by the research team to determine eligibility for thromboprophylaxis. Main Outcomes and Measures: The primary outcome was rate of HA-VTE. Secondary outcomes included rates of prophylactic anticoagulation and anticoagulation-associated bleeding events. Results: A total of 17 427 hospitalizations met eligibility criteria, were randomized, and were included in the primary analysis: patients had a median (IQR) age of 1.7 (0 to 11.1) years; there were 9143 (52.5%) female patients and 8284 (47.5%) male patients, and there were 445 (2.6%) Asian patients, 2739 (15.9%) Black patients, and 11 752 (67.4%) White patients. The 2 groups were evenly balanced in number (8717 in the intervention group and 8710 in the control group) and patient characteristics. A total of 58 patients (0.7%) in the control group and 77 (0.9%) in the intervention group developed HA-VTE (risk difference: 2.2 per 1000 patients; 95% CI, -0.4 to 4.8 per 1000 patients; P = .10). Recommendations to initiate thromboprophylaxis were accepted by primary clinical teams 25.8% of the time (74 of 287 hospitalizations). Minor bleeding events were rare among patients who received anticoagulation (3 of 74 [4.1%]), and no major bleeding events were observed during the study period. Among patients randomized to the control group, the model exhibited high discrimination accuracy (C statistic, 0.799, 95% CI, 0.725 to 0.856). Conclusions and Relevance: In this randomized clinical trial of the use of a HA-VTE prognostic model to reduce pediatric inpatient rates of HA-VTE, despite the use of an accurate and validated prognostic model for HA-VTE, there was substantial reluctance by primary clinical teams to initiate thromboprophylaxis as recommended. In this context, rates of HA-VTE between the control and intervention groups were not different. Future research is needed to identify improved strategies for prevention of HA-VTE and to overcome clinician concerns regarding thromboprophylaxis. Trial Registration: ClinicalTrials.gov Identifier: NCT04574895.

Protocol for a randomised controlled trial: reducing reintubation among high-risk cardiac surgery patients with high-flow nasal cannula (I-CAN).

INTRODUCTION: Heated, humidified, high-flow nasal cannula oxygen therapy has been used as a therapy for hypoxic respiratory failure in numerous clinical settings. To date, limited data exist to guide appropriate use following cardiac surgery, particularly among patients at risk for experiencing reintubation. We hypothesised that postextubation treatment with high-flow nasal cannula would decrease the all-cause reintubation rate within the 48 hours following initial extubation, compared with usual care. METHODS AND ANALYSIS: Adult patients undergoing cardiac surgery (open surgery on the heart or thoracic aorta) will be automatically enrolled, randomised and allocated to one of two treatment arms in a pragmatic randomised controlled trial at the time of initial extubation. The two treatment arms are administration of heated, humidified, high-flow nasal cannula oxygen postextubation and usual care (treatment at the discretion of the treating provider). The primary outcome will be all-cause reintubation within 48 hours of initial extubation. Secondary outcomes include all-cause 30-day mortality, hospital length of stay, intensive care unit length of stay and ventilator-free days. Interaction analyses will be conducted to assess the differential impact of the intervention within strata of predicted risk of reintubation, calculated according to our previously published and validated prognostic model. ETHICS AND DISSEMINATION: Vanderbilt University Medical Center IRB approval, 15 March 2021 with waiver of written informed consent. Plan for publication of study protocol prior to study completion, as well as publication of results. TRIAL REGISTRATION NUMBER: clinicaltrials.gov, NCT04782817 submitted 25 February 2021. DATE OF PROTOCOL: 29 August 2022. Version 2.0.

A Predictive Model of Reintubation After Cardiac Surgery Using the Electronic Health Record.

BACKGROUND: Reintubation and prolonged intubation after cardiac surgery are associated with significant complications. Despite these competing risks, providers frequently extubate patients with limited insight into the risk of reintubation at the time of extubation. Achieving timely, successful extubation remains a significant clinical challenge. METHODS: Based on an analysis of 2835 patients undergoing cardiac surgery at our institution between November 2017 and July 2020, we developed a model for an individual's risk of reintubation at the time of extubation. Predictors were screened for inclusion in the model based on clinical plausibility and availability at the time of extubation. Rigorous data reduction methods were used to create a model that could be easily integrated into clinical workflow at the time of extubation. RESULTS: In total, 90 patients (3.2%) were reintubated within 48 hours of initial extubation. Number of inotropes (1 [adjusted odds ratio (OR), 15.4; 95% confidence interval (CI) 6.5-47.6; P < .001], ≥2 [OR, 62.7; 95% CI 14.3-279.5; P < .001]); dexmedetomidine dose (OR, 3.0 [per µg/kg/h]; 95% CI 1.9-4.7; P < .001), time to extubation (OR, 1.04 [per 6-hour increase]; 95% CI 1.02-1.05; P < .001), and respiratory rate (OR, 1.04 [per breath/min]; 95% CI 1.01-1.07; P < .001) were the best predictors for the model, which displayed excellent discriminative capacity (area under the receiver operating characteristic, 0.86; 95% CI 0.84-0.89). CONCLUSIONS: An improved understanding of reintubation risk may lead to improved decision-making at extubation and targeted interventions to decrease reintubation in high-risk patients. Future studies are needed to optimize timing of extubation.

The first 1000 symptomatic pediatric SARS-CoV-2 infections in an integrated health care system: a prospective cohort study.

BACKGROUND: The spectrum of illness and predictors of severity among children with SARS-CoV-2 infection are incompletely understood. METHODS: Active surveillance was performed for SARS-CoV-2 by polymerase chain reaction among symptomatic pediatric patients in a quaternary care academic hospital laboratory beginning March 12, 2020. We obtained sociodemographic and clinical data 5 (+/-3) and 30 days after diagnosis via phone follow-up and medical record review. Logistic regression was used to assess predictors of hospitalization. RESULTS: The first 1000 symptomatic pediatric patients were diagnosed in our institution between March 13, 2020 and September 28, 2020. Cough (52 %), headache (43 %), and sore throat (36 %) were the most common symptoms. Forty-one (4 %) were hospitalized; 8 required ICU admission, and 2 required mechanical ventilation (< 1 %). One patient developed multisystem inflammatory syndrome in children; one death was possibly associated with SARS-CoV-2 infection. Symptom resolution occurred by follow-up day 5 in 398/892 (45 %) patients and by day 30 in 443/471 (94 %) patients. Pre-existing medical condition (OR 7.7; 95 % CI 3.9-16.0), dyspnea (OR 6.8; 95 % CI 3.2-14.1), Black race or Hispanic ethnicity (OR 2.7; 95 % CI 1.3-5.5), and vomiting (OR 5.4; 95 % CI 1.2-20.6) were the strongest predictors of hospitalization. The model displayed excellent discriminative ability (AUC = 0.82, 95 % CI 0.76-0.88, Brier score = 0.03). CONCLUSIONS: In 1000 pediatric patients with systematic follow-up, most SARS-CoV-2 infections were mild, brief, and rarely required hospitalization. Pediatric predictors of hospitalization included comorbid conditions, Black race, Hispanic ethnicity, dyspnea and vomiting and were distinct from those reported among adults.