ABSTRACT
OBJECTIVES: To develop an electronic descriptor of clinical deterioration for hospitalized patients that predicts short-term mortality and identifies patient deterioration earlier than current standard definitions. DESIGN: A retrospective study using exploratory record review, quantitative analysis, and regression analyses. SETTING: Twelve-hospital community-academic health system. PATIENTS: All adult patients with an acute hospital encounter between January 1, 2018, and December 31, 2022. INTERVENTIONS: Not applicable. MEASUREMENTS AND MAIN RESULTS: Clinical trigger events were selected and used to create a revised electronic definition of deterioration, encompassing signals of respiratory failure, bleeding, and hypotension occurring in proximity to ICU transfer. Patients meeting the revised definition were 12.5 times more likely to die within 7 days (adjusted odds ratio 12.5; 95% CI, 8.9-17.4) and had a 95.3% longer length of stay (95% CI, 88.6-102.3%) compared with those who were transferred to the ICU or died regardless of meeting the revised definition. Among the 1812 patients who met the revised definition of deterioration before ICU transfer (52.4%), the median detection time was 157.0 min earlier (interquartile range 64.0-363.5 min). CONCLUSIONS: The revised definition of deterioration establishes an electronic descriptor of clinical deterioration that is strongly associated with short-term mortality and length of stay and identifies deterioration over 2.5 hours earlier than ICU transfer. Incorporating the revised definition of deterioration into the training and validation of early warning system algorithms may enhance their timeliness and clinical accuracy.
ABSTRACT
Importance: The Deterioration Index (DTI), used by hospitals for predicting patient deterioration, has not been extensively validated externally, raising concerns about performance and equitable predictions. Objective: To locally validate DTI performance and assess its potential for bias in predicting patient clinical deterioration. Design, Setting, and Participants: This retrospective prognostic study included 13 737 patients admitted to 8 heterogenous Midwestern US hospitals varying in size and type, including academic, community, urban, and rural hospitals. Patients were 18 years or older and admitted between January 1 and May 31, 2021. Exposure: DTI predictions made every 15 minutes. Main Outcomes and Measures: Deterioration, defined as the occurrence of any of the following while hospitalized: mechanical ventilation, intensive care unit transfer, or death. Performance of the DTI was evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC). Bias measures were calculated across demographic subgroups. Results: A total of 5â¯143â¯513 DTI predictions were made for 13â¯737 patients across 14â¯834 hospitalizations. Among 13 918 encounters, the mean (SD) age of patients was 60.3 (19.2) years; 7636 (54.9%) were female, 11 345 (81.5%) were White, and 12 392 (89.0%) were of other ethnicity than Hispanic or Latino. The prevalence of deterioration was 10.3% (n = 1436). The DTI produced AUROCs of 0.759 (95% CI, 0.756-0.762) at the observation level and 0.685 (95% CI, 0.671-0.700) at the encounter level. Corresponding AUPRCs were 0.039 (95% CI, 0.037-0.040) at the observation level and 0.248 (95% CI, 0.227-0.273) at the encounter level. Bias measures varied across demographic subgroups and were 14.0% worse for patients identifying as American Indian or Alaska Native and 19.0% worse for those who chose not to disclose their ethnicity. Conclusions and Relevance: In this prognostic study, the DTI had modest ability to predict patient deterioration, with varying degrees of performance at the observation and encounter levels and across different demographic groups. Disparate performance across subgroups suggests the need for more transparency in model training data and reinforces the need to locally validate externally developed prediction models.
