Development and Validation of a Machine Learning COVID-19 Veteran (COVet) Deterioration Risk Score.

BACKGROUND AND OBJECTIVE: To develop the COVid Veteran (COVet) score for clinical deterioration in Veterans hospitalized with COVID-19 and further validate this model in both Veteran and non-Veteran samples. No such score has been derived and validated while incorporating a Veteran sample. DERIVATION COHORT: Adults (age ≥ 18 yr) hospitalized outside the ICU with a diagnosis of COVID-19 for model development to the Veterans Health Administration (VHA) (n = 80 hospitals). VALIDATION COHORT: External validation occurred in a VHA cohort of 34 hospitals, as well as six non-Veteran health systems for further external validation (n = 21 hospitals) between 2020 and 2023. PREDICTION MODEL: eXtreme Gradient Boosting machine learning methods were used, and performance was assessed using the area under the receiver operating characteristic curve and compared with the National Early Warning Score (NEWS). The primary outcome was transfer to the ICU or death within 24 hours of each new variable observation. Model predictor variables included demographics, vital signs, structured flowsheet data, and laboratory values. RESULTS: A total of 96,908 admissions occurred during the study period, of which 59,897 were in the Veteran sample and 37,011 were in the non-Veteran sample. During external validation in the Veteran sample, the model demonstrated excellent discrimination, with an area under the receiver operating characteristic curve of 0.88. This was significantly higher than NEWS (0.79; p < 0.01). In the non-Veteran sample, the model also demonstrated excellent discrimination (0.86 vs. 0.79 for NEWS; p < 0.01). The top three variables of importance were eosinophil percentage, mean oxygen saturation in the prior 24-hour period, and worst mental status in the prior 24-hour period. CONCLUSIONS: We used machine learning methods to develop and validate a highly accurate early warning score in both Veterans and non-Veterans hospitalized with COVID-19. The model could lead to earlier identification and therapy, which may improve outcomes.

National Early Warning Score Deployment in a Veterans Affairs Facility: A Quality Improvement Initiative and Analysis.

Early warning scores are algorithms designed to identify clinical deterioration. Current literature is predominantly in non-Veteran populations. Studies in Veterans are lacking. This study was a prospective quality improvement project deploying and assessing the National Early Warning Score (NEWS) at Kansas City VA Medical Center. Performance of NEWS was assessed as follows: discrimination for predicting a composite outcome of intensive care unit transfer or mortality within 24 hours via area under the receiver operating curve. A total of 4781 Veterans with 142 375 NEWS values were included. The NEWS area under the receiver operating curve for the composite outcome was 0.72 (95% CI, 0.71-0.74), indicating acceptable predictive accuracy. A NEWS of ≥7 was more likely associated with the composite outcome versus <7 (13.6% vs 0.8%; P < 0.001). This is one of the first studies to demonstrate successful deployment of NEWS in a Veteran population, with resultant important implications across the Veterans Health Administration.