Prospective validation of clinical deterioration predictive models prior to intensive care unit transfer among patients admitted to acute care cardiology wards.

Objective. Very few predictive models have been externally validated in a prospective cohort following the implementation of an artificial intelligence analytic system. This type of real-world validation is critically important due to the risk of data drift, or changes in data definitions or clinical practices over time, that could impact model performance in contemporaneous real-world cohorts. In this work, we report the model performance of a predictive analytics tool developed before COVID-19 and demonstrate model performance during the COVID-19 pandemic.Approach. The analytic system (CoMETⓇ, Nihon Kohden Digital Health Solutions LLC, Irvine, CA) was implemented in a randomized controlled trial that enrolled 10 422 patient visits in a 1:1 display-on display-off design. The CoMET scores were calculated for all patients but only displayed in the display-on arm. Only the control/display-off group is reported here because the scores could not alter care patterns.Main results.Of the 5184 visits in the display-off arm, 311 experienced clinical deterioration and care escalation, resulting in transfer to the intensive care unit, primarily due to respiratory distress. The model performance of CoMET was assessed based on areas under the receiver operating characteristic curve, which ranged from 0.725 to 0.737.Significance.The models were well-calibrated, and there were dynamic increases in the model scores in the hours preceding the clinical deterioration events. A hypothetical alerting strategy based on a rise in score and duration of the rise would have had good performance, with a positive predictive value more than 10-fold the event rate. We conclude that predictive statistical models developed five years before study initiation had good model performance despite the passage of time and the impact of the COVID-19 pandemic.

Overt and Occult Hypoxemia in Patients Hospitalized With COVID-19.

Progressive hypoxemia is the predominant mode of deterioration in COVID-19. Among hypoxemia measures, the ratio of the Pao2 to the Fio2 (P/F ratio) has optimal construct validity but poor availability because it requires arterial blood sampling. Pulse oximetry reports oxygenation continuously (ratio of the Spo2 to the Fio2 [S/F ratio]), but it is affected by skin color and occult hypoxemia can occur in Black patients. Oxygen dissociation curves allow noninvasive estimation of P/F ratios (ePFRs) but remain unproven. OBJECTIVES: Measure overt and occult hypoxemia using ePFR. DESIGN SETTING AND PARTICIPANTS: We retrospectively studied COVID-19 hospital encounters (n = 5,319) at two academic centers (University of Virginia [UVA] and Emory University). MAIN OUTCOMES AND MEASURES: We measured primary outcomes (death or ICU transfer within 24 hr), ePFR, conventional hypoxemia measures, baseline predictors (age, sex, race, comorbidity), and acute predictors (National Early Warning Score [NEWS] and Sequential Organ Failure Assessment [SOFA]). We updated predictors every 15 minutes. We assessed predictive validity using adjusted odds ratios (AORs) and area under the receiver operating characteristic curves (AUROCs). We quantified disparities (Black vs non-Black) in empirical cumulative distributions using the Kolmogorov-Smirnov (K-S) two-sample test. RESULTS: Overt hypoxemia (low ePFR) predicted bad outcomes (AOR for a 100-point ePFR drop: 2.7 [UVA]; 1.7 [Emory]; p < 0.01) with better discrimination (AUROC: 0.76 [UVA]; 0.71 [Emory]) than NEWS (0.70 [both sites]) or SOFA (0.68 [UVA]; 0.65 [Emory]) and similar to S/F ratio (0.76 [UVA]; 0.70 [Emory]). We found racial differences consistent with occult hypoxemia. Black patients had better apparent oxygenation (K-S distance: 0.17 [both sites]; p < 0.01) but, for comparable ePFRs, worse outcomes than other patients (AOR: 2.2 [UVA]; 1.2 [Emory]; p < 0.01). CONCLUSIONS AND RELEVANCE: The ePFR was a valid measure of overt hypoxemia. In COVID-19, it may outperform multi-organ dysfunction models. By accounting for biased oximetry as well as clinicians' real-time responses to it (supplemental oxygen adjustment), ePFRs may reveal racial disparities attributable to occult hypoxemia.

Beyond prediction: Off-target uses of artificial intelligence-based predictive analytics in a learning health system.

Introduction: Artificial-intelligence (AI)-based predictive analytics provide new opportunities to leverage rich sources of continuous data to improve patient care through early warning of the risk of clinical deterioration and improved situational awareness.Part of the success of predictive analytic implementation relies on integration of the analytic within complex clinical workflows. Pharmaceutical interventions have off-target uses where a drug indication has not been formally studied for a different indication but has potential for clinical benefit. An analog has not been described in the context of AI-based predictive analytics, that is, when a predictive analytic has been trained on one outcome of interest but is used for additional applications in clinical practice. Methods: In this manuscript we present three clinical vignettes describing off-target use of AI-based predictive analytics that evolved organically through real-world practice. Results: Off-target uses included:real-time feedback about treatment effectiveness, indication of readiness to discharge, and indication of the acuity of a hospital unit. Conclusion: Such practice fits well with the learning health system goals to continuously integrate data and experience to provide.

Accuracy of a Risk Alert Threshold for ICU Hypoglycemia: Retrospective Analysis of Alert Performance and Association With Clinical Deterioration Events.

OBJECTIVES: To quantify the accuracy of and clinical events associated with a risk alert threshold for impending hypoglycemia during ICU admissions. DESIGN: Retrospective electronic health record review of clinical events occurring greater than or equal to 1 and less than or equal to 12 hours after the hypoglycemia risk alert threshold was met. SETTING: Adult ICU admissions from June 2020 through April 2021 at the University of Virginia Medical Center. PATIENTS: Three hundred forty-two critically ill adults that were 63.5% male with median age 60.8 years, median weight 79.1 kg, and median body mass index of 27.5 kg/m2. INTERVENTIONS: Real-world testing of our validated predictive model as a clinical decision support tool for ICU hypoglycemia. MEASUREMENTS AND MAIN RESULTS: We retrospectively reviewed 350 hypothetical alerts that met inclusion criteria for analysis. The alerts correctly predicted 48 cases of level 1 hypoglycemia that occurred greater than or equal to 1 and less than or equal to 12 hours after the alert threshold was met (positive predictive value = 13.7%). Twenty-one of these 48 cases (43.8%) involved level 2 hypoglycemia. Notably, three myocardial infarctions, one medical emergency team call, 19 deaths, and 20 arrhythmias occurred greater than or equal to 1 and less than or equal to 12 hours after an alert threshold was met. CONCLUSIONS: Alerts generated by a validated ICU hypoglycemia prediction model had a positive predictive value of 13.7% for real-world hypoglycemia events. This proof-of-concept result suggests that the predictive model offers clinical value, but further prospective testing is needed to confirm this.

A Diagnostic Prediction Model to Distinguish Multisystem Inflammatory Syndrome in Children.

OBJECTIVE: Features of multisystem inflammatory syndrome in children (MIS-C) overlap with other syndromes, making the diagnosis difficult for clinicians. We aimed to compare clinical differences between patients with and without clinical MIS-C diagnosis and develop a diagnostic prediction model to assist clinicians in identification of patients with MIS-C within the first 24 hours of hospital presentation. METHODS: A cohort of 127 patients (<21 years) were admitted to an academic children's hospital and evaluated for MIS-C. The primary outcome measure was MIS-C diagnosis at Vanderbilt University Medical Center. Clinical, laboratory, and cardiac features were extracted from the medical record, compared among groups, and selected a priori to identify candidate predictors. Final predictors were identified through a logistic regression model with bootstrapped backward selection in which only variables selected in more than 80% of 500 bootstraps were included in the final model. RESULTS: Of 127 children admitted to our hospital with concern for MIS-C, 45 were clinically diagnosed with MIS-C and 82 were diagnosed with alternative diagnoses. We found a model with four variables-the presence of hypotension and/or fluid resuscitation, abdominal pain, new rash, and the value of serum sodium-showed excellent discrimination (concordance index 0.91; 95% confidence interval: 0.85-0.96) and good calibration in identifying patients with MIS-C. CONCLUSION: A diagnostic prediction model with early clinical and laboratory features shows excellent discrimination and may assist clinicians in distinguishing patients with MIS-C. This model will require external and prospective validation prior to widespread use.

Signatures of illness in children requiring unplanned intubation in the pediatric intensive care unit: A retrospective cohort machine-learning study.

Acute respiratory failure requiring the initiation of invasive mechanical ventilation remains commonplace in the pediatric intensive care unit (PICU). Early recognition of patients at risk for respiratory failure may provide clinicians with the opportunity to intervene and potentially improve outcomes. Through the development of a random forest model to identify patients at risk for requiring unplanned intubation, we tested the hypothesis that subtle signatures of illness are present in physiological and biochemical time series of PICU patients in the early stages of respiratory decompensation. We included 116 unplanned intubation events as recorded in the National Emergency Airway Registry for Children in 92 PICU admissions over a 29-month period at our institution. We observed that children have a physiologic signature of illness preceding unplanned intubation in the PICU. Generally, it comprises younger age, and abnormalities in electrolyte, hematologic and vital sign parameters. Additionally, given the heterogeneity of the PICU patient population, we found differences in the presentation among the major patient groups - medical, cardiac surgical, and non-cardiac surgical. At four hours prior to the event, our random forest model demonstrated an area under the receiver operating characteristic curve of 0.766 (0.738 for medical, 0.755 for cardiac surgical, and 0.797 for non-cardiac surgical patients). The multivariable statistical models that captured the physiological and biochemical dynamics leading up to the event of urgent unplanned intubation in a PICU can be repurposed for bedside risk prediction.

Overt and occult hypoxemia in patients hospitalized with novel coronavirus disease 2019.

Background: Progressive hypoxemia is the predominant mode of deterioration in COVID-19. Among hypoxemia measures, the ratio of the partial pressure of arterial oxygen to the fraction of inspired oxygen (P/F ratio) has optimal construct validity but poor availability because it requires arterial blood sampling. Pulse oximetry reports oxygenation continuously, but occult hypoxemia can occur in Black patients because the technique is affected by skin color. Oxygen dissociation curves allow non-invasive estimation of P/F ratios (ePFR) but this approach remains unproven. Research Question: Can ePFRs measure overt and occult hypoxemia? Study Design and methods: We retrospectively studied COVID-19 hospital encounters (n=5319) at two academic centers (University of Virginia [UVA] and Emory University). We measured primary outcomes (death or ICU transfer within 24 hours), ePFR, conventional hypoxemia measures, baseline predictors (age, sex, race, comorbidity), and acute predictors (National Early Warning Score (NEWS) and Sepsis-3). We updated predictors every 15 minutes. We assessed predictive validity using adjusted odds ratios (AOR) and area under receiver operating characteristics curves (AUROC). We quantified disparities (Black vs non-Black) in empirical cumulative distributions using the Kolmogorov-Smirnov (K-S) two-sample test. Results: Overt hypoxemia (low ePFR) predicted bad outcomes (AOR for a 100-point ePFR drop: 2.7 [UVA]; 1.7 [Emory]; p<0.01) with better discrimination (AUROC: 0.76 [UVA]; 0.71 [Emory]) than NEWS (AUROC: 0.70 [UVA]; 0.70 [Emory]) or Sepsis-3 (AUROC: 0.68 [UVA]; 0.65 [Emory]). We found racial differences consistent with occult hypoxemia. Black patients had better apparent oxygenation (K-S distance: 0.17 [both sites]; p<0.01) but, for comparable ePFRs, worse outcomes than other patients (AOR: 2.2 [UVA]; 1.2 [Emory], p<0.01). Interpretation: The ePFR was a valid measure of overt hypoxemia. In COVID-19, it may outperform multi-organ dysfunction models like NEWS and Sepsis-3. By accounting for biased oximetry as well as clinicians’ real-time responses to it (supplemental oxygen adjustment), ePFRs may enable statistical modelling of racial disparities in outcomes attributable to occult hypoxemia.

Pathophysiologic Signature of Impending ICU Hypoglycemia in Bedside Monitoring and Electronic Health Record Data: Model Development and External Validation.

OBJECTIVES: We tested the hypothesis that routine monitoring data could describe a detailed and distinct pathophysiologic phenotype of impending hypoglycemia in adult ICU patients. DESIGN: Retrospective analysis leading to model development and validation. SETTING: All ICU admissions wherein patients received insulin therapy during a 4-year period at the University of Virginia Medical Center. Each ICU was equipped with continuous physiologic monitoring systems whose signals were archived in an electronic data warehouse along with the entire medical record. PATIENTS: Eleven thousand eight hundred forty-seven ICU patient admissions. INTERVENTIONS: The primary outcome was hypoglycemia, defined as any episode of blood glucose less than 70 mg/dL where 50% dextrose injection was administered within 1 hour. We used 61 physiologic markers (including vital signs, laboratory values, demographics, and continuous cardiorespiratory monitoring variables) to inform the model. MEASUREMENTS AND MAIN RESULTS: Our dataset consisted of 11,847 ICU patient admissions, 721 (6.1%) of which had one or more hypoglycemic episodes. Multivariable logistic regression analysis revealed a pathophysiologic signature of 41 independent variables that best characterized ICU hypoglycemia. The final model had a cross-validated area under the receiver operating characteristic curve of 0.83 (95% CI, 0.78-0.87) for prediction of impending ICU hypoglycemia. We externally validated the model in the Medical Information Mart for Intensive Care III critical care dataset, where it also demonstrated good performance with an area under the receiver operating characteristic curve of 0.79 (95% CI, 0.77-0.81). CONCLUSIONS: We used data from a large number of critically ill inpatients to develop and externally validate a predictive model of impending ICU hypoglycemia. Future steps include incorporating this model into a clinical decision support system and testing its effects in a multicenter randomized controlled clinical trial.

External validation of a novel signature of illness in continuous cardiorespiratory monitoring to detect early respiratory deterioration of ICU patients.

OBJECTIVE: The goal of predictive analytics monitoring is the early detection of patients at high risk of subacute potentially catastrophic illnesses. An excellent example of a targeted illness is respiratory failure leading to urgent unplanned intubation, where early detection might lead to interventions that improve patient outcomes. Previously, we identified signatures of this illness in the continuous cardiorespiratory monitoring data of intensive care unit (ICU) patients and devised algorithms to identify patients at rising risk. Here, we externally validated three logistic regression models to estimate the risk of emergency intubation developed in Medical and Surgical ICUs at the University of Virginia. APPROACH: We calculated the model outputs for more than 8000 patients in the University of California-San Francisco ICUs, 240 of whom underwent emergency intubation as determined by individual chart review. MAIN RESULTS: We found that the AUC of the models exceeded 0.75 in this external population, and that the risk rose appreciably over the 12 h before the event. SIGNIFICANCE: We conclude that there are generalizable physiological signatures of impending respiratory failure in the continuous cardiorespiratory monitoring data.

Predictive Monitoring-Impact in Acute Care Cardiology Trial (PM-IMPACCT): Protocol for a Randomized Controlled Trial.

BACKGROUND: Patients in acute care wards who deteriorate and are emergently transferred to intensive care units (ICUs) have poor outcomes. Early identification of patients who are decompensating might allow for earlier clinical intervention and reduced morbidity and mortality. Advances in bedside continuous predictive analytics monitoring (ie, artificial intelligence [AI]-based risk prediction) have made complex data easily available to health care providers and have provided early warning of potentially catastrophic clinical events. We present a dynamic, visual, predictive analytics monitoring tool that integrates real-time bedside telemetric physiologic data into robust clinical models to estimate and communicate risk of imminent events. This tool, Continuous Monitoring of Event Trajectories (CoMET), has been shown in retrospective observational studies to predict clinical decompensation on the acute care ward. There is a need to more definitively study this advanced predictive analytics or AI monitoring system in a prospective, randomized controlled, clinical trial. OBJECTIVE: The goal of this trial is to determine the impact of an AI-based visual risk analytic, CoMET, on improving patient outcomes related to clinical deterioration, response time to proactive clinical action, and costs to the health care system. METHODS: We propose a cluster randomized controlled trial to test the impact of using the CoMET display in an acute care cardiology and cardiothoracic surgery hospital floor. The number of admissions to a room undergoing cluster randomization was estimated to be 10,424 over the 20-month study period. Cluster randomization based on bed number will occur every 2 months. The intervention cluster will have the CoMET score displayed (along with standard of care), while the usual care group will receive standard of care only. RESULTS: The primary outcome will be hours free from events of clinical deterioration. Hours of acute clinical events are defined as time when one or more of the following occur: emergent ICU transfer, emergent surgery prior to ICU transfer, cardiac arrest prior to ICU transfer, emergent intubation, or death. The clinical trial began randomization in January 2021. CONCLUSIONS: Very few AI-based health analytics have been translated from algorithm to real-world use. This study will use robust, prospective, randomized controlled, clinical trial methodology to assess the effectiveness of an advanced AI predictive analytics monitoring system in incorporating real-time telemetric data for identifying clinical deterioration on acute care wards. This analysis will strengthen the ability of health care organizations to evolve as learning health systems, in which bioinformatics data are applied to improve patient outcomes by incorporating AI into knowledge tools that are successfully integrated into clinical practice by health care providers. TRIAL REGISTRATION: ClinicalTrials.gov NCT04359641; https://clinicaltrials.gov/ct2/show/NCT04359641. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/29631.

Dynamic data in the ED predict requirement for ICU transfer following acute care admission.

Misidentification of illness severity may lead to patients being admitted to a ward bed then unexpectedly transferring to an ICU as their condition deteriorates. Our objective was to develop a predictive analytic tool to identify emergency department (ED) patients that required upgrade to an intensive or intermediate care unit (ICU or IMU) within 24 h after being admitted to an acute care floor. We conducted a single-center retrospective cohort study to identify ED patients that were admitted to an acute care unit and identified cases where the patient was upgraded to ICU or IMU within 24 h. We used data available at the time of admission to build a logistic regression model that predicts early ICU transfer. We found 42,332 patients admitted between January 2012 and December 2016. There were 496 cases (1.2%) of early ICU transfer. Case patients had 18.0-fold higher mortality (11.1% vs. 0.6%, p < 0.001) and 3.4 days longer hospital stays (5.9 vs. 2.5, p < 0.001) than those without an early transfer. Our predictive analytic model had a cross-validated area under the receiver operating characteristic of 0.70 (95% CI 0.67-0.72) and identified 10% of early ICU transfers with an alert rate of 1.6 per week (162.2 acute care admits per week, 1.9 early ICU transfers). Predictive analytic monitoring based on data available in the emergency department can identify patients that will require upgrade to ICU or IMU if admitted to acute care. Incorporating this tool into ED practice may draw attention to high-risk patients before acute care admit and allow early intervention.

Pathophysiologic Signatures of Bloodstream Infection in Critically Ill Adults.

OBJECTIVES: Bloodstream infection is associated with high mortality rates in critically ill patients but is difficult to identify clinically. This results in frequent blood culture testing, exposing patients to additional costs as well as the potential harms of unnecessary antibiotics. The purpose of this study was to assess whether the analysis of bedside physiologic monitoring data could accurately describe a pathophysiologic signature of bloodstream infection in patients admitted to the ICU. DESIGN: Development of a statistical model using physiologic data from a retrospective observational cohort. SETTING: University of Virginia Medical Center (Charlottesville, VA), a tertiary-care academic medical center. PATIENTS: Critically ill patients consecutively admitted to either the medical or surgical/trauma ICUs with available physiologic monitoring data between February 2011 and June 2015. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We analyzed 9,954 ICU admissions with 144 patient-years of vital sign and electrocardiography waveform data, totaling 1.3 million hourly measurements. There were 15,577 blood culture instances, with 1,184 instances of bloodstream infection (8%). The multivariate pathophysiologic signature of bloodstream infection was characterized by abnormalities in 15 different physiologic features. The cross-validated area under the receiver operating characteristic curve was 0.78 (95% CI, 0.69-0.85). We also identified distinct signatures of Gram-negative and fungal bloodstream infections, but not Gram-positive bloodstream infection. CONCLUSIONS: Signatures of bloodstream infection can be identified in the routine physiologic monitoring data of critically ill adults. This may assist in identifying infected patients, maximizing diagnostic stewardship, and measuring the effect of new therapeutic modalities for sepsis.

Early Detection of In-Patient Deterioration: One Prediction Model Does Not Fit All.

OBJECTIVES: Early detection of subacute potentially catastrophic illnesses using available data is a clinical imperative, and scores that report risk of imminent events in real time abound. Patients deteriorate for a variety of reasons, and it is unlikely that a single predictor such as an abnormal National Early Warning Score will detect all of them equally well. The objective of this study was to test the idea that the diversity of reasons for clinical deterioration leading to ICU transfer mandates multiple targeted predictive models. DESIGN: Individual chart review to determine the clinical reason for ICU transfer; determination of relative risks of individual vital signs, laboratory tests and cardiorespiratory monitoring measures for prediction of each clinical reason for ICU transfer; and logistic regression modeling for the outcome of ICU transfer for a specific clinical reason. SETTING: Cardiac medical-surgical ward; tertiary care academic hospital. PATIENTS: Eight-thousand one-hundred eleven adult patients, 457 of whom were transferred to an ICU for clinical deterioration. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: We calculated the contributing relative risks of individual vital signs, laboratory tests and cardiorespiratory monitoring measures for prediction of each clinical reason for ICU transfer, and used logistic regression modeling to calculate receiver operating characteristic areas and relative risks for the outcome of ICU transfer for a specific clinical reason. The reasons for clinical deterioration leading to ICU transfer were varied, as were their predictors. For example, the three most common reasons-respiratory instability, infection and suspected sepsis, and heart failure requiring escalated therapy-had distinct signatures of illness. Statistical models trained to target-specific reasons for ICU transfer performed better than one model targeting combined events. CONCLUSIONS: A single predictive model for clinical deterioration does not perform as well as having multiple models trained for the individual specific clinical events leading to ICU transfer.

Towards development of alert thresholds for clinical deterioration using continuous predictive analytics monitoring.

Patients who deteriorate while on the acute care ward and are emergently transferred to the Intensive Care Unit (ICU) experience high rates of mortality. To date, risk scores for clinical deterioration applied to the acute care wards rely on static or intermittent inputs of vital sign and assessment parameters. We propose the use of continuous predictive analytics monitoring, or data that relies on real-time physiologic monitoring data captured from ECG, documented vital signs, laboratory results, and other clinical assessments to predict clinical deterioration. A necessary step in translation to practice is understanding how an alert threshold would perform if applied to a continuous predictive analytic that was trained to detect clinical deterioration. The purpose of this study was to evaluate the positive predictive value of 'risk spikes', or large abrupt increases in the output of a statistical model of risk predicting clinical deterioration. We studied 8111 consecutive patient admissions to a cardiovascular medicine and surgery ward with continuous ECG data. We first trained a multivariable logistic regression model for emergent ICU transfer in a test set and tested the characteristics of the model in a validation set of 4059 patient admissions. Then, in a nested analysis we identified large, abrupt spikes in risk (increase by three units over the prior 6 h; a unit is the fold-increase in risk of ICU transfer in the next 24 h) and reviewed hospital records of 91 patients for clinical events such as emergent ICU transfer. We compared results to 59 control patients at times when they were matched for baseline risk including the National Warning Score (NEWS). There was a 3.4-fold higher event rate for patients with risk spikes (positive predictive value 24% compared to 7%, p = 0.006). If we were to use risk spikes as an alert, they would fire about once per day on a 73-bed acute care ward. Risk spikes that were primarily driven by respiratory changes (ECG-derived respiration (EDR) or charted respiratory rate) had highest PPV (30-35%) while risk spikes driven by heart rate had the lowest (7%). Alert thresholds derived from continuous predictive analytics monitoring are able to be operationalized as a degree of change from the person's own baseline rather than arbitrary threshold cut-points, which can likely better account for the individual's own inherent acuity levels. Point of care clinicians in the acute care ward settings need tailored alert strategies that promote a balance in recognition of clinical deterioration and assessment of the utility of the alert approach.

Predictive analytics in the pediatric intensive care unit for early identification of sepsis: capturing the context of age.

BACKGROUND: Early recognition of patients at risk for sepsis is paramount to improve clinical outcomes. We hypothesized that subtle signatures of illness are present in physiological and biochemical time series of pediatric-intensive care unit (PICU) patients in the early stages of sepsis. METHODS: We developed multivariate models in a retrospective observational cohort to predict the clinical diagnosis of sepsis in children. We focused on age as a predictor and asked whether random forest models, with their potential for multiple cut points, had better performance than logistic regression. RESULTS: One thousand seven hundred and eleven admissions for 1425 patients admitted to a mixed cardiac and medical/surgical PICU were included. We identified, through individual chart review, 187 sepsis diagnoses that were not within 14 days of a prior sepsis diagnosis. Multivariate models predicted sepsis in the next 24 h: cross-validated C-statistic for logistic regression and random forest were 0.74 (95% confidence interval (CI): 0.71-0.77) and 0.76 (95% CI: 0.73-0.79), respectively. CONCLUSIONS: Statistical models based on physiological and biochemical data already available in the PICU identify high-risk patients up to 24 h prior to the clinical diagnosis of sepsis. The random forest model was superior to logistic regression in capturing the context of age.

Impact of predictive analytics based on continuous cardiorespiratory monitoring in a surgical and trauma intensive care unit.

Predictive analytics monitoring, the use of patient data to provide continuous risk estimation of deterioration, is a promising new application of big data analytical techniques to the care of individual patients. We tested the hypothesis that continuous display of novel electronic risk visualization of respiratory and cardiovascular events would impact intensive care unit (ICU) patient outcomes. In an adult tertiary care surgical trauma ICU, we displayed risk estimation visualizations on a large monitor, but in the medical ICU in the same institution we did not. The risk estimates were based solely on analysis of continuous cardiorespiratory monitoring. We examined 4275 individual patient records within a 7 month time period preceding and following data display. We determined cases of septic shock, emergency intubation, hemorrhage, and death to compare rates per patient care pre-and post-implementation. Following implementation, the incidence of septic shock fell by half (p < 0.01 in a multivariate model that included age and APACHE) in the surgical trauma ICU, where the data were continuously on display, but by only 10% (p = NS) in the control Medical ICU. There were no significant changes in the other outcomes. Display of a predictive analytics monitor based on continuous cardiorespiratory monitoring was followed by a reduction in the rate of septic shock, even when controlling for age and APACHE score.

Dynamic data monitoring improves predictive analytics for failed extubation in the ICU.

OBJECTIVE: Predictive analytics monitoring that informs clinicians of the risk for failed extubation would help minimize both the duration of mechanical ventilation and the risk of emergency re-intubation in ICU patients. We hypothesized that dynamic monitoring of cardiorespiratory data, vital signs, and lab test results would add information to standard clinical risk factors. METHODS: We report model development in a retrospective observational cohort admitted to either the medical or surgical/trauma ICU that were intubated during their ICU stay and had available physiologic monitoring data (n = 1202). The primary outcome was removal of endotracheal intubation (i.e. extubation) followed within 48 h by reintubation or death (i.e. failed extubation). We developed a standard risk marker model based on demographic and clinical data. We also developed a novel risk marker model using dynamic data elements-continuous cardiorespiratory monitoring, vital signs, and lab values. RESULTS: Risk estimates from multivariate predictive models in the 24 h preceding extubation were significantly higher for patients that failed. Combined standard and novel risk markers demonstrated good predictive performance in leave-one-out validation: AUC of 0.64 (95% CI: 0.57-0.69) and 1.6 alerts per week to identify 32% of extubations that will fail. Novel risk factors added significantly to the standard model. CONCLUSION: Predictive analytics monitoring models can detect changes in vital signs, continuous cardiorespiratory monitoring, and laboratory measurements in both the hours preceding and following extubation for those patients destined for extubation failure.

Advancing Continuous Predictive Analytics Monitoring: Moving from Implementation to Clinical Action in a Learning Health System.

In the intensive care unit, clinicians monitor a diverse array of data inputs to detect early signs of impending clinical demise or improvement. Continuous predictive analytics monitoring synthesizes data from a variety of inputs into a risk estimate that clinicians can observe in a streaming environment. For this to be useful, clinicians must engage with the data in a way that makes sense for their clinical workflow in the context of a learning health system (LHS). This article describes the processes needed to evoke clinical action after initiation of continuous predictive analytics monitoring in an LHS.

Identifying the low risk patient in surgical intensive and intermediate care units using continuous monitoring.

BACKGROUND: Continuous predictive monitoring has been employed successfully to predict subclinical adverse events. Should low values on these models, however, reassure us that a patient will not have an adverse outcome? Negative predictive values of such models could help predict safe patient discharge. The goal of this study was to validate the negative predictive value of an ensemble model for critical illness (using previously developed models for respiratory instability, hemorrhage, and sepsis) based on bedside monitoring data in the intensive care units and intermediate care unit. METHODS: We calculated the relative risk of 3 critical illnesses for all patients every 15 minutes (n= 124,588) for 2,924 patients downgraded from the surgical intensive care units and intermediate care unit between May 2014 to May 2016. We constructed an ensemble model to estimate at the time of intensive care units or intermediate care unit discharge the probability of favorable outcome after downgrade. RESULTS: Outputs form the ensemble model stratified patients by risk of favorable and bad outcomes in both intensive care units/intermediate care unit; area under the receiver operating characteristic curve = .639/.629 respectively for favorable outcomes and .645/.641 for adverse events. These performance characteristics are commensurate with published models for predicting readmission. The ensemble model remained a statistically significant predictor after adjusting for hospital duration of stay and admitting service. The rate of favorable outcome in the highest and lowest deciles in the intensive care units were 76.2% and 27.3% (2.8-fold decrease) and 88.3% and 33.2% in the intermediate care unit (2.7-fold decrease), respectively. CONCLUSION: An ensemble model for critical illness predicts favorable outcome after downgrade and safe patient discharge (hospital stay <7 days, no readmission, upgrade, or death).

Cardiorespiratory dynamics measured from continuous ECG monitoring improves detection of deterioration in acute care patients: A retrospective cohort study.

BACKGROUND: Charted vital signs and laboratory results represent intermittent samples of a patient's dynamic physiologic state and have been used to calculate early warning scores to identify patients at risk of clinical deterioration. We hypothesized that the addition of cardiorespiratory dynamics measured from continuous electrocardiography (ECG) monitoring to intermittently sampled data improves the predictive validity of models trained to detect clinical deterioration prior to intensive care unit (ICU) transfer or unanticipated death. METHODS AND FINDINGS: We analyzed 63 patient-years of ECG data from 8,105 acute care patient admissions at a tertiary care academic medical center. We developed models to predict deterioration resulting in ICU transfer or unanticipated death within the next 24 hours using either vital signs, laboratory results, or cardiorespiratory dynamics from continuous ECG monitoring and also evaluated models using all available data sources. We calculated the predictive validity (C-statistic), the net reclassification improvement, and the probability of achieving the difference in likelihood ratio χ2 for the additional degrees of freedom. The primary outcome occurred 755 times in 586 admissions (7%). We analyzed 395 clinical deteriorations with continuous ECG data in the 24 hours prior to an event. Using only continuous ECG measures resulted in a C-statistic of 0.65, similar to models using only laboratory results and vital signs (0.63 and 0.69 respectively). Addition of continuous ECG measures to models using conventional measurements improved the C-statistic by 0.01 and 0.07; a model integrating all data sources had a C-statistic of 0.73 with categorical net reclassification improvement of 0.09 for a change of 1 decile in risk. The difference in likelihood ratio χ2 between integrated models with and without cardiorespiratory dynamics was 2158 (p value: <0.001). CONCLUSIONS: Cardiorespiratory dynamics from continuous ECG monitoring detect clinical deterioration in acute care patients and improve performance of conventional models that use only laboratory results and vital signs.