Telemedicine Critical Care-Mediated Mortality Reductions in Lower-Performing Patient Diagnosis Groups: A Prospective, Before and After Study.

OBJECTIVES: Studies evaluating telemedicine critical care (TCC) have shown mixed results. We prospectively evaluated the impact of TCC implementation on risk-adjusted mortality among patients stratified by pre-TCC performance. DESIGN: Prospective, observational, before and after study. SETTING: Three adult ICUs at an academic medical center. PATIENTS: A total of 2,429 patients in the pre-TCC (January to June 2016) and 12,479 patients in the post-TCC (January 2017 to June 2019) periods. INTERVENTIONS: TCC implementation which included an acuity-driven workflow targeting an identified "lower-performing" patient group, defined by ICU admission in an Acute Physiology and Chronic Health Evaluation diagnoses category with a pre-TCC standardized mortality ratio (SMR) of greater than 1.5. MEASUREMENTS AND MAIN RESULTS: The primary outcome was risk-adjusted hospital mortality. Risk-adjusted hospital length of stay (HLOS) was also studied. The SMR for the overall ICU population was 0.83 pre-TCC and 0.75 post-TCC, with risk-adjusted mortalities of 10.7% and 9.5% (p = 0.09). In the identified lower-performing patient group, which accounted for 12.6% (n = 307) of pre-TCC and 13.3% (n = 1671) of post-TCC ICU patients, SMR decreased from 1.61 (95% CI, 1.21-2.01) pre-TCC to 1.03 (95% CI, 0.91-1.15) post-TCC, and risk-adjusted mortality decreased from 26.4% to 16.9% (p < 0.001). In the remaining ("higher-performing") patient group, there was no change in pre- versus post-TCC SMR (0.70 [0.59-0.81] vs 0.69 [0.64-0.73]) or risk-adjusted mortality (8.5% vs 8.4%, p = 0.86). There were no pre- to post-TCC differences in standardized HLOS ratio or risk-adjusted HLOS in the overall cohort or either performance group. CONCLUSIONS: In well-staffed and overall higher-performing ICUs in an academic medical center, Acute Physiology and Chronic Health Evaluation granularity allowed identification of a historically lower-performing patient group that experienced a striking TCC-associated reduction in SMR and risk-adjusted mortality. This study provides additional evidence for the relationship between pre-TCC performance and post-TCC improvement.

Inter-Rater Reliability and Impact of Disagreements on Acute Physiology and Chronic Health Evaluation IV Mortality Predictions.

Acute Physiology and Chronic Health Evaluation is a well-validated method to risk-adjust ICU patient outcomes. However, predictions may be affected by inter-rater reliability for manually entered elements. We evaluated inter-rater reliability for Acute Physiology and Chronic Health Evaluation IV manually entered elements among clinician abstractors and assessed the impacts of disagreements on mortality predictions. DESIGN: Cross-sectional. SETTING: Academic medical center. SUBJECTS: Patients admitted to five adult ICUs. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Acute Physiology and Chronic Health Evaluation IV manually entered elements were abstracted from a selection of charts (n = 41) by two clinician "raters" trained in Acute Physiology and Chronic Health Evaluation IV methodology. Rater agreement (%) was determined for each manually entered element, including Acute Physiology and Chronic Health Evaluation diagnosis, Glasgow Coma Scale score, admission source, chronic conditions, elective/emergency surgery, and ventilator use. Cohen's kappa (K) or intraclass correlation coefficient was calculated for nominal and continuous manually entered elements, respectively. The impacts of manually entered element choices on Acute Physiology and Chronic Health Evaluation IV mortality predictions were computed using published Acute Physiology and Chronic Health Evaluation IV equations, and observed to expected hospital mortality ratios were compared between rater groups. The majority of manually entered element inconsistency was due to disagreement in choice of Glasgow Coma Scale (63.8% agreement, 0.83 intraclass correlation coefficient), Acute Physiology and Chronic Health Evaluation diagnosis (68.3% agreement, 0.67 kappa), and admission source (90.2% agreement, 0.85 kappa). The difference in predicted mortality between raters related to Glasgow Coma Scale disagreements was significant (observed to expected mortality ratios for Rater 1 [1.009] vs Rater 2 [1.134]; p < 0.05). Differences related to Acute Physiology and Chronic Health Evaluation diagnosis or admission source disagreements were negligible. The new "unable to score" choice for Glasgow Coma Scale was used for 18% of Glasgow Coma Scale measurements but accounted for 63% of "major" Glasgow Coma Scale disagreements, and 50% of the overall difference in Acute Physiology and Chronic Health Evaluation-predicted mortality between raters. CONCLUSIONS: Inconsistent use among raters of the new "unable to score" choice for Glasgow Coma Scale introduced in Acute Physiology and Chronic Health Evaluation IV was responsible for important decreases in both Glasgow Coma Scale and Acute Physiology and Chronic Health Evaluation IV mortality prediction reliability in our study. A Glasgow Coma Scale algorithm we developed after the study to improve reliability related to use of this new "unable to score" choice is presented.