Early Warning Systems for Critical Illness Outside the Intensive Care Unit.

Early warning systems (EWSs) are designed and deployed to create a rapid assessment and response for patients with clinical deterioration outside the intensive care unit (ICU). These models incorporate patient-level data such as vital signs and laboratory values to detect or prevent adverse clinical events, such as vital signs and laboratories to allow detection and prevention of adverse clinical events such as cardiac arrest, intensive care transfer, or sepsis. The applicability, development, clinical utility, and general perception of EWS in clinical practice vary widely. Here, we review the field as it has grown from early vital sign-based scoring systems to contemporary multidimensional algorithms and predictive technologies for clinical decompensation outside the ICU.

Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis.

Early recognition and treatment of sepsis are linked to improved patient outcomes. Machine learning-based early warning systems may reduce the time to recognition, but few systems have undergone clinical evaluation. In this prospective, multi-site cohort study, we examined the association between patient outcomes and provider interaction with a deployed sepsis alert system called the Targeted Real-time Early Warning System (TREWS). During the study, 590,736 patients were monitored by TREWS across five hospitals. We focused our analysis on 6,877 patients with sepsis who were identified by the alert before initiation of antibiotic therapy. Adjusting for patient presentation and severity, patients in this group whose alert was confirmed by a provider within 3 h of the alert had a reduced in-hospital mortality rate (3.3%, confidence interval (CI) 1.7, 5.1%, adjusted absolute reduction, and 18.7%, CI 9.4, 27.0%, adjusted relative reduction), organ failure and length of stay compared with patients whose alert was not confirmed by a provider within 3 h. Improvements in mortality rate (4.5%, CI 0.8, 8.3%, adjusted absolute reduction) and organ failure were larger among those patients who were additionally flagged as high risk. Our findings indicate that early warning systems have the potential to identify sepsis patients early and improve patient outcomes and that sepsis patients who would benefit the most from early treatment can be identified and prioritized at the time of the alert.

Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing.

Machine learning-based clinical decision support tools for sepsis create opportunities to identify at-risk patients and initiate treatments at early time points, which is critical for improving sepsis outcomes. In view of the increasing use of such systems, better understanding of how they are adopted and used by healthcare providers is needed. Here, we analyzed provider interactions with a sepsis early detection tool (Targeted Real-time Early Warning System), which was deployed at five hospitals over a 2-year period. Among 9,805 retrospectively identified sepsis cases, the early detection tool achieved high sensitivity (82% of sepsis cases were identified) and a high rate of adoption: 89% of all alerts by the system were evaluated by a physician or advanced practice provider and 38% of evaluated alerts were confirmed by a provider. Adjusting for patient presentation and severity, patients with sepsis whose alert was confirmed by a provider within 3 h had a 1.85-h (95% CI 1.66-2.00) reduction in median time to first antibiotic order compared to patients with sepsis whose alert was either dismissed, confirmed more than 3 h after the alert or never addressed in the system. Finally, we found that emergency department providers and providers who had previous interactions with an alert were more likely to interact with alerts, as well as to confirm alerts on retrospectively identified patients with sepsis. Beyond efforts to improve the performance of early warning systems, efforts to improve adoption are essential to their clinical impact and should focus on understanding providers' knowledge of, experience with and attitudes toward such systems.

Human-machine teaming is key to AI adoption: clinicians' experiences with a deployed machine learning system.

While a growing number of machine learning (ML) systems have been deployed in clinical settings with the promise of improving patient care, many have struggled to gain adoption and realize this promise. Based on a qualitative analysis of coded interviews with clinicians who use an ML-based system for sepsis, we found that, rather than viewing the system as a surrogate for their clinical judgment, clinicians perceived themselves as partnering with the technology. Our findings suggest that, even without a deep understanding of machine learning, clinicians can build trust with an ML system through experience, expert endorsement and validation, and systems designed to accommodate clinicians' autonomy and support them across their entire workflow.

Comparison of Automated Sepsis Identification Methods and Electronic Health Record-based Sepsis Phenotyping: Improving Case Identification Accuracy by Accounting for Confounding Comorbid Conditions.

To develop and evaluate a novel strategy that automates the retrospective identification of sepsis using electronic health record data. DESIGN: Retrospective cohort study of emergency department and in-hospital patient encounters from 2014 to 2018. SETTING: One community and two academic hospitals in Maryland. PATIENTS: All patients 18 years old or older presenting to the emergency department or admitted to any acute inpatient medical or surgical unit including patients discharged from the emergency department. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: From the electronic health record, 233,252 emergency department and inpatient encounters were identified. Patient data were used to develop and validate electronic health record-based sepsis phenotyping, an adaptation of "the Centers for Disease Control Adult Sepsis Event toolkit" that accounts for comorbid conditions when identifying sepsis patients. The performance of this novel system was then compared with 1) physician case review and 2) three other commonly used strategies using metrics of sensitivity and precision relative to sepsis billing codes, termed "billing code sensitivity" and "billing code predictive value." Physician review of electronic health record-based sepsis phenotyping identified cases confirmed 79% as having sepsis; 88% were confirmed or had a billing code for sepsis; and 99% were confirmed, had a billing code, or received at least 4 days of antibiotics. At comparable billing code sensitivity (0.91; 95% CI, 0.88-0.93), electronic health record-based sepsis phenotyping had a higher billing code predictive value (0.32; 95% CI, 0.30-0.34) than either the Centers for Medicare and Medicaid Services Sepsis Core Measure (SEP-1) definition or the Sepsis-3 consensus definition (0.12; 95% CI, 0.11-0.13; and 0.07; 95% CI, 0.07-0.08, respectively). When compared with electronic health record-based sepsis phenotyping, Adult Sepsis Event had a lower billing code sensitivity (0.75; 95% CI, 0.72-0.78) and similar billing code predictive value (0.29; 95% CI, 0.26-0.31). Electronic health record-based sepsis phenotyping identified patients with higher in-hospital mortality and nearly one-half as many false-positive cases when compared with SEP-1 and Sepsis-3. CONCLUSIONS: By accounting for comorbid conditions, electronic health record-based sepsis phenotyping exhibited better performance when compared with other automated definitions of sepsis.

A targeted real-time early warning score (TREWScore) for septic shock.

Sepsis is a leading cause of death in the United States, with mortality highest among patients who develop septic shock. Early aggressive treatment decreases morbidity and mortality. Although automated screening tools can detect patients currently experiencing severe sepsis and septic shock, none predict those at greatest risk of developing shock. We analyzed routinely available physiological and laboratory data from intensive care unit patients and developed "TREWScore," a targeted real-time early warning score that predicts which patients will develop septic shock. TREWScore identified patients before the onset of septic shock with an area under the ROC (receiver operating characteristic) curve (AUC) of 0.83 [95% confidence interval (CI), 0.81 to 0.85]. At a specificity of 0.67, TREWScore achieved a sensitivity of 0.85 and identified patients a median of 28.2 [interquartile range (IQR), 10.6 to 94.2] hours before onset. Of those identified, two-thirds were identified before any sepsis-related organ dysfunction. In comparison, the Modified Early Warning Score, which has been used clinically for septic shock prediction, achieved a lower AUC of 0.73 (95% CI, 0.71 to 0.76). A routine screening protocol based on the presence of two of the systemic inflammatory response syndrome criteria, suspicion of infection, and either hypotension or hyperlactatemia achieved a lower sensitivity of 0.74 at a comparable specificity of 0.64. Continuous sampling of data from the electronic health records and calculation of TREWScore may allow clinicians to identify patients at risk for septic shock and provide earlier interventions that would prevent or mitigate the associated morbidity and mortality.