ABSTRACT
BACKGROUND: Despite its high lethality, sepsis can be difficult to detect on initial presentation to the emergency department (ED). Machine learning-based tools may provide avenues for earlier detection and lifesaving intervention. OBJECTIVE: The study aimed to predict sepsis at the time of ED triage using natural language processing of nursing triage notes and available clinical data. METHODS: We constructed a retrospective cohort of all 1,234,434 consecutive ED encounters in 2015-2021 from 4 separate clinically heterogeneous academically affiliated EDs. After exclusion criteria were applied, the final cohort included 1,059,386 adult ED encounters. The primary outcome criteria for sepsis were presumed severe infection and acute organ dysfunction. After vectorization and dimensional reduction of triage notes and clinical data available at triage, a decision tree-based ensemble (time-of-triage) model was trained to predict sepsis using the training subset (n=950,921). A separate (comprehensive) model was trained using these data and laboratory data, as it became available at 1-hour intervals, after triage. Model performances were evaluated using the test (n=108,465) subset. RESULTS: Sepsis occurred in 35,318 encounters (incidence 3.45%). For sepsis prediction at the time of patient triage, using the primary definition, the area under the receiver operating characteristic curve (AUC) and macro F1-score for sepsis were 0.94 and 0.61, respectively. Sensitivity, specificity, and false positive rate were 0.87, 0.85, and 0.15, respectively. The time-of-triage model accurately predicted sepsis in 76% (1635/2150) of sepsis cases where sepsis screening was not initiated at triage and 97.5% (1630/1671) of cases where sepsis screening was initiated at triage. Positive and negative predictive values were 0.18 and 0.99, respectively. For sepsis prediction using laboratory data available each hour after ED arrival, the AUC peaked to 0.97 at 12 hours. Similar results were obtained when stratifying by hospital and when Centers for Disease Control and Prevention hospital toolkit for adult sepsis surveillance criteria were used to define sepsis. Among septic cases, sepsis was predicted in 36.1% (1375/3814), 49.9% (1902/3814), and 68.3% (2604/3814) of encounters, respectively, at 3, 2, and 1 hours prior to the first intravenous antibiotic order or where antibiotics where not ordered within the first 12 hours. CONCLUSIONS: Sepsis can accurately be predicted at ED presentation using nursing triage notes and clinical information available at the time of triage. This indicates that machine learning can facilitate timely and reliable alerting for intervention. Free-text data can improve the performance of predictive modeling at the time of triage and throughout the ED course.
ABSTRACT
OBJECTIVE: Accurate triage in the emergency department (ED) is critical for medical safety and operational efficiency. We aimed to predict the number of future required ED resources, as defined by the Emergency Severity Index (ESI) triage protocol, using natural language processing of nursing triage notes. METHODS: We constructed a retrospective cohort of all 265,572 consecutive ED encounters from 2015 to 2016 from 3 separate clinically heterogeneous academically affiliated EDs. We excluded encounters missing relevant information, leaving 226,317 encounters. We calculated the number of resources used by patients in the ED retrospectively and based outcome categories on criteria defined in the ESI algorithm: 0 (30,604 encounters), 1 (49,315 encounters), and 2 or more (146,398 encounters). A neural network model was trained on a training subset to predict the number of resources using triage notes and clinical variables at triage. Model performance was evaluated using the test subset and was compared with human ratings. RESULTS: Overall model accuracy and macro F1 score for number of resources were 66.5% and 0.601, respectively. The model had similar macro F1 (0.589 vs 0.592) and overall accuracy (65.9% vs 69.0%) compared to human raters. Model predictions had slightly higher F1 scores and accuracy for 0 resources and were less accurate for 2 or more resources. CONCLUSIONS: Machine learning of nursing triage notes, combined with clinical data available at ED presentation, can be used to predict the number of required future ED resources. These findings suggest that machine learning may be a valuable adjunct tool in the initial triage of ED patients.
