ABSTRACT
BACKGROUND: Successful integrations of machine learning into routine clinical care are exceedingly rare, and barriers to its adoption are poorly characterized in the literature. OBJECTIVE: This study aims to report a quality improvement effort to integrate a deep learning sepsis detection and management platform, Sepsis Watch, into routine clinical care. METHODS: In 2016, a multidisciplinary team consisting of statisticians, data scientists, data engineers, and clinicians was assembled by the leadership of an academic health system to radically improve the detection and treatment of sepsis. This report of the quality improvement effort follows the learning health system framework to describe the problem assessment, design, development, implementation, and evaluation plan of Sepsis Watch. RESULTS: Sepsis Watch was successfully integrated into routine clinical care and reshaped how local machine learning projects are executed. Frontline clinical staff were highly engaged in the design and development of the workflow, machine learning model, and application. Novel machine learning methods were developed to detect sepsis early, and implementation of the model required robust infrastructure. Significant investment was required to align stakeholders, develop trusting relationships, define roles and responsibilities, and to train frontline staff, leading to the establishment of 3 partnerships with internal and external research groups to evaluate Sepsis Watch. CONCLUSIONS: Machine learning models are commonly developed to enhance clinical decision making, but successful integrations of machine learning into routine clinical care are rare. Although there is no playbook for integrating deep learning into clinical care, learnings from the Sepsis Watch integration can inform efforts to develop machine learning technologies at other health care delivery systems.
ABSTRACT
INTRODUCTION: Triage systems play a vital role in emergency department (ED) operations and can determine how well a given ED serves its local population. We sought to describe ED utilization patterns for different triage levels using the National Hospital Ambulatory Medical Care Survey (NHAMCS) database. METHODS: We conducted a multi-year secondary analysis of the NHAMCS database from 2009-2011. National visit estimates were made using standard methods in Analytics Software and Solutions (SAS, Cary, NC). We compared patients in the mid-urgency range in regard to ED lengths of stay, hospital admission rates, and numbers of tests and procedures in comparison to lower or higher acuity levels. RESULTS: We analyzed 100,962 emergency visits (representing 402,211,907 emergency visits nationwide). In 2011, patients classified as triage levels 1-3 had a higher number of diagnoses (5.5, 5.6 and 4.2, respectively) when compared to those classified as levels 4 and 5 (1.61 and 1.25). This group also underwent a higher number of procedures (1.0, 0.8 and 0.7, versus 0.4 and 0.4), had a higher ED length of stay (220, 280 and 237, vs. 157 and 135), and admission rates (32.2%, 32.3% and 15.5%, vs. 3.1% and 3.6%). CONCLUSION: Patients classified as mid-level (3) triage urgency require more resources and have higher indicators of acuity as those in triage levels 4 and 5. These patients' indicators are more similar to those classified as triage levels 1 and 2.
