Evaluating the Effect of a COVID-19 Predictive Model to Facilitate Discharge: A Randomized Controlled Trial.

BACKGROUND: We previously developed and validated a predictive model to help clinicians identify hospitalized adults with coronavirus disease 2019 (COVID-19) who may be ready for discharge given their low risk of adverse events. Whether this algorithm can prompt more timely discharge for stable patients in practice is unknown. OBJECTIVES: The aim of the study is to estimate the effect of displaying risk scores on length of stay (LOS). METHODS: We integrated model output into the electronic health record (EHR) at four hospitals in one health system by displaying a green/orange/red score indicating low/moderate/high-risk in a patient list column and a larger COVID-19 summary report visible for each patient. Display of the score was pseudo-randomized 1:1 into intervention and control arms using a patient identifier passed to the model execution code. Intervention effect was assessed by comparing LOS between intervention and control groups. Adverse safety outcomes of death, hospice, and re-presentation were tested separately and as a composite indicator. We tracked adoption and sustained use through daily counts of score displays. RESULTS: Enrolling 1,010 patients from May 15, 2020 to December 7, 2020, the trial found no detectable difference in LOS. The intervention had no impact on safety indicators of death, hospice or re-presentation after discharge. The scores were displayed consistently throughout the study period but the study lacks a causally linked process measure of provider actions based on the score. Secondary analysis revealed complex dynamics in LOS temporally, by primary symptom, and hospital location. CONCLUSION: An AI-based COVID-19 risk score displayed passively to clinicians during routine care of hospitalized adults with COVID-19 was safe but had no detectable impact on LOS. Health technology challenges such as insufficient adoption, nonuniform use, and provider trust compounded with temporal factors of the COVID-19 pandemic may have contributed to the null result. TRIAL REGISTRATION: ClinicalTrials.gov identifier: NCT04570488.

Six-Month Outcomes in Patients Hospitalized with Severe COVID-19.

BACKGROUND: Previous work has demonstrated that patients experience functional decline at 1-3 months post-discharge after COVID-19 hospitalization. OBJECTIVE: To determine whether symptoms persist further or improve over time, we followed patients discharged after hospitalization for severe COVID-19 to characterize their overall health status and their physical and mental health at 6 months post-hospital discharge. DESIGN: Prospective observational cohort study. PARTICIPANTS: Patients ≥ 18 years hospitalized for COVID-19 at a single health system, who required at minimum 6 l of supplemental oxygen during admission, had intact baseline functional status, and were discharged alive. MAIN MEASURES: Overall health status, physical health, mental health, and dyspnea were assessed with validated surveys: the PROMIS® Global Health-10 and PROMIS® Dyspnea Characteristics instruments. KEY RESULTS: Of 152 patients who completed the 1 month post-discharge survey, 126 (83%) completed the 6-month survey. Median age of 6-month respondents was 62; 40% were female. Ninety-three (74%) patients reported that their health had not returned to baseline at 6 months, and endorsed a mean of 7.1 symptoms. Participants' summary t-scores in both the physical health and mental health domains at 6 months (45.2, standard deviation [SD] 9.8; 47.4, SD 9.8, respectively) remained lower than their baseline (physical health 53.7, SD 9.4; mental health 54.2, SD 8.0; p<0.001). Overall, 79 (63%) patients reported shortness of breath within the prior week (median score 2 out of 10 (interquartile range [IQR] 0-5), vs 42 (33%) pre-COVID-19 infection (0, IQR 0-1)). A total of 11/124 (9%) patients without pre-COVID oxygen requirements still needed oxygen 6 months post-hospital discharge. One hundred and seven (85%) were still experiencing fatigue at 6 months post-discharge. CONCLUSIONS: Even 6 months after hospital discharge, the majority of patients report that their health has not returned to normal. Support and treatments to return these patients back to their pre-COVID baseline are urgently needed.

Applying A/B Testing to Clinical Decision Support: Rapid Randomized Controlled Trials.

BACKGROUND: Clinical decision support (CDS) is a valuable feature of electronic health records (EHRs) designed to improve quality and safety. However, due to the complexities of system design and inconsistent results, CDS tools may inadvertently increase alert fatigue and contribute to physician burnout. A/B testing, or rapid-cycle randomized tests, is a useful method that can be applied to the EHR in order to rapidly understand and iteratively improve design choices embedded within CDS tools. OBJECTIVE: This paper describes how rapid randomized controlled trials (RCTs) embedded within EHRs can be used to quickly ascertain the superiority of potential CDS design changes to improve their usability, reduce alert fatigue, and promote quality of care. METHODS: A multistep process combining tools from user-centered design, A/B testing, and implementation science was used to understand, ideate, prototype, test, analyze, and improve each candidate CDS. CDS engagement metrics (alert views, acceptance rates) were used to evaluate which CDS version is superior. RESULTS: To demonstrate the impact of the process, 2 experiments are highlighted. First, after multiple rounds of usability testing, a revised CDS influenza alert was tested against usual care CDS in a rapid (~6 weeks) RCT. The new alert text resulted in minimal impact on reducing firings per patients per day, but this failure triggered another round of review that identified key technical improvements (ie, removal of dismissal button and firings in procedural areas) that led to a dramatic decrease in firings per patient per day (23.1 to 7.3). In the second experiment, the process was used to test 3 versions (financial, quality, regulatory) of text supporting tobacco cessation alerts as well as 3 supporting images. Based on 3 rounds of RCTs, there was no significant difference in acceptance rates based on the framing of the messages or addition of images. CONCLUSIONS: These experiments support the potential for this new process to rapidly develop, deploy, and rigorously evaluate CDS within an EHR. We also identified important considerations in applying these methods. This approach may be an important tool for improving the impact of and experience with CDS. TRIAL REGISTRATION: Flu alert trial: ClinicalTrials.gov NCT03415425; https://clinicaltrials.gov/ct2/show/NCT03415425. Tobacco alert trial: ClinicalTrials.gov NCT03714191; https://clinicaltrials.gov/ct2/show/NCT03714191.

Post-Discharge Health Status and Symptoms in Patients with Severe COVID-19.

BACKGROUND: Little is known about long-term recovery from severe COVID-19 disease. Here, we characterize overall health, physical health, and mental health of patients 1 month after discharge for severe COVID-19. METHODS: This was a prospective single health system observational cohort study of patients ≥ 18 years hospitalized with laboratory-confirmed COVID-19 disease who required at least 6 l of oxygen during admission, had intact baseline cognitive and functional status, and were discharged alive. Participants were enrolled between 30 and 40 days after discharge. Outcomes were elicited through validated survey instruments: the PROMIS® Dyspnea Characteristics and PROMIS® Global Health-10. RESULTS: A total of 161 patients (40.6% of eligible) were enrolled; 152 (38.3%) completed the survey. Median age was 62 years (interquartile range [IQR], 50-67); 57 (37%) were female. Overall, 113/152 (74%) participants reported shortness of breath within the prior week (median score 3 out of 10 [IQR 0-5]), vs 47/152 (31%) pre-COVID-19 infection (0, IQR 0-1), p < 0.001. Participants also rated their physical health and mental health as worse in their post-COVID state (43.8, standard deviation 9.3; mental health 47.3, SD 9.3) compared to their pre-COVID state, (54.3, SD 9.3; 54.3, SD 7.8, respectively), both p < 0.001. Physical and mental health means in the general US population are 50 (SD 10). A total of 52/148 (35.1%) patients without pre-COVID oxygen requirements needed home oxygen after hospital discharge; 20/148 (13.5%) reported still using oxygen at time of survey. CONCLUSIONS: Patients with severe COVID-19 disease typically experience sequelae affecting their respiratory status, physical health, and mental health for at least several weeks after hospital discharge.

Post-discharge health status and symptoms in patients with severe COVID-19.

BACKGROUND: Little is known about long-term recovery from severe COVID-19 disease. Here, we characterize overall health, physical health and mental health of patients one month after discharge for severe COVID-19. METHODS: This was a prospective single health system observational cohort study of patients ≥18 years hospitalized with laboratory-confirmed COVID-19 disease who required at least 6 liters of oxygen during admission, had intact baseline cognitive and functional status and were discharged alive. Participants were enrolled between 30 and 40 days after discharge. Outcomes were elicited through validated survey instruments: the PROMIS Dyspnea Characteristics and PROMIS Global Health-10. RESULTS: A total of 161 patients (40.6% of eligible) were enrolled; 152 (38.3%) completed the survey. Median age was 62 years (interquartile range [IQR], 50-67); 57 (37%) were female. Overall, 113/152 (74%) participants reported shortness of breath within the prior week (median score 3 out of 10 [IQR 0-5]), vs. 47/152 (31%) pre-COVID-19 infection (0, IQR 0-1), p<0.001. Participants also rated their physical health and mental health as worse in their post-COVID state (43.8, standard deviation 9.3; mental health 47.3, SD 9.3) compared to their pre-COVID state, (54.3, SD 9.3; 54.3, SD 7.8, respectively), both p <0.001. A total of 52/148 (35.1%) patients without pre-COVID oxygen requirements needed home oxygen after hospital discharge; 20/148 (13.5%) reported still using oxygen at time of survey. CONCLUSIONS: Patients with severe COVID-19 disease typically experience sequelae affecting their respiratory status, physical health and mental health for at least several weeks after hospital discharge.