Improving documentation of presenting problems in the emergency department using a domain-specific ontology and machine learning-driven user interfaces.

OBJECTIVES: To determine the effect of a domain-specific ontology and machine learning-driven user interfaces on the efficiency and quality of documentation of presenting problems (chief complaints) in the emergency department (ED). METHODS: As part of a quality improvement project, we simultaneously implemented three interventions: a domain-specific ontology, contextual autocomplete, and top five suggestions. Contextual autocomplete is a user interface that ranks concepts by their predicted probability which helps nurses enter data about a patient's presenting problems. Nurses were also given a list of top five suggestions to choose from. These presenting problems were represented using a consensus ontology mapped to SNOMED CT. Predicted probabilities were calculated using a previously derived model based on triage vital signs and a brief free text note. We evaluated the percentage and quality of structured data captured using a mixed methods retrospective before-and-after study design. RESULTS: A total of 279,231 consecutive patient encounters were analyzed. Structured data capture improved from 26.2% to 97.2% (p < 0.0001). During the post-implementation period, presenting problems were more complete (3.35 vs 3.66; p = 0.0004) and higher in overall quality (3.38 vs. 3.72; p = 0.0002), but showed no difference in precision (3.59 vs. 3.74; p = 0.1). Our system reduced the mean number of keystrokes required to document a presenting problem from 11.6 to 0.6 (p < 0.0001), a 95% improvement. DISCUSSION: We demonstrated a technique that captures structured data on nearly all patients. We estimate that our system reduces the number of man-hours required annually to type presenting problems at our institution from 92.5 h to 4.8 h. CONCLUSION: Implementation of a domain-specific ontology and machine learning-driven user interfaces resulted in improved structured data capture, ontology usage compliance, and data quality.

Creating an automated trigger for sepsis clinical decision support at emergency department triage using machine learning.

OBJECTIVE: To demonstrate the incremental benefit of using free text data in addition to vital sign and demographic data to identify patients with suspected infection in the emergency department. METHODS: This was a retrospective, observational cohort study performed at a tertiary academic teaching hospital. All consecutive ED patient visits between 12/17/08 and 2/17/13 were included. No patients were excluded. The primary outcome measure was infection diagnosed in the emergency department defined as a patient having an infection related ED ICD-9-CM discharge diagnosis. Patients were randomly allocated to train (64%), validate (20%), and test (16%) data sets. After preprocessing the free text using bigram and negation detection, we built four models to predict infection, incrementally adding vital signs, chief complaint, and free text nursing assessment. We used two different methods to represent free text: a bag of words model and a topic model. We then used a support vector machine to build the prediction model. We calculated the area under the receiver operating characteristic curve to compare the discriminatory power of each model. RESULTS: A total of 230,936 patient visits were included in the study. Approximately 14% of patients had the primary outcome of diagnosed infection. The area under the ROC curve (AUC) for the vitals model, which used only vital signs and demographic data, was 0.67 for the training data set, 0.67 for the validation data set, and 0.67 (95% CI 0.65-0.69) for the test data set. The AUC for the chief complaint model which also included demographic and vital sign data was 0.84 for the training data set, 0.83 for the validation data set, and 0.83 (95% CI 0.81-0.84) for the test data set. The best performing methods made use of all of the free text. In particular, the AUC for the bag-of-words model was 0.89 for training data set, 0.86 for the validation data set, and 0.86 (95% CI 0.85-0.87) for the test data set. The AUC for the topic model was 0.86 for the training data set, 0.86 for the validation data set, and 0.85 (95% CI 0.84-0.86) for the test data set. CONCLUSION: Compared to previous work that only used structured data such as vital signs and demographic information, utilizing free text drastically improves the discriminatory ability (increase in AUC from 0.67 to 0.86) of identifying infection.