Effect of Machine Learning on Anaesthesiology Clinician Prediction of Postoperative Complications: The Perioperative ORACLE Randomised Clinical Trial.

Background: Anaesthesiology clinicians can implement risk mitigation strategies if they know which patients are at greatest risk for postoperative complications. Although machine learning models predicting complications exist, their impact on clinician risk assessment is unknown. Methods: This single-centre randomised clinical trial enrolled patients age ≥18 undergoing surgery with anaesthesiology services. Anaesthesiology clinicians providing remote intraoperative telemedicine support reviewed electronic health records with (assisted group) or without (unassisted group) also reviewing machine learning predictions. Clinicians predicted the likelihood of postoperative 30-day all-cause mortality and postoperative acute kidney injury within 7 days. Area under the receiver operating characteristic curve (AUROC) for the clinician predictions was determined. Results: Among 5,071 patient cases reviewed by 89 clinicians, the observed incidence was 2% for postoperative death and 11% for acute kidney injury. Clinician predictions agreed with the models more strongly in the assisted versus unassisted group (weighted kappa 0.75 versus 0.62 for death [difference 0.13, 95%CI 0.10-0.17] and 0.79 versus 0.54 for kidney injury [difference 0.25, 95%CI 0.21-0.29]). Clinicians predicted death with AUROC of 0.793 in the assisted group and 0.780 in the unassisted group (difference 0.013, 95%CI -0.070 to 0.097). Clinicians predicted kidney injury with AUROC of 0.734 in the assisted group and 0.688 in the unassisted group (difference 0.046, 95%CI -0.003 to 0.091). Conclusions: Although there was evidence that the models influenced clinician predictions, clinician performance was not statistically significantly different with and without machine learning assistance. Further work is needed to clarify the role of machine learning in real-time perioperative risk stratification. Trial Registration: ClinicalTrials.gov NCT05042804.

Effect of Telemedicine Support for Intraoperative Anaesthesia Care on Postoperative Outcomes: The TECTONICS Randomised Clinical Trial.

Background: Novel applications of telemedicine can improve care quality and patient outcomes. Telemedicine for intraoperative decision support has not been rigorously studied. Methods: This single centre randomised clinical trial ( clinicaltrials.gov NCT03923699 ) of unselected adult surgical patients was conducted between July 1, 2019 and January 31, 2023. Patients received usual care or decision support from a telemedicine service, the Anesthesiology Control Tower (ACT). The ACT provided real-time recommendations to intraoperative anaesthesia clinicians based on case reviews, machine-learning forecasting, and physiologic alerts. ORs were randomised 1:1. Co-primary outcomes of 30-day all-cause mortality, respiratory failure, acute kidney injury (AKI), and delirium were analysed as intention-to-treat. Results: The trial completed planned enrolment with 71927 surgeries (35956 ACT; 35971 usual care). After multiple testing correction, there was no significant effect of the ACT vs. usual care on 30-day mortality [641/35956 (1.8%) vs 638/35971 (1.8%), risk difference 0.0% (95% CI -0.2% to 0.3%), p=0.96], respiratory failure [1089/34613 (3.1%) vs 1112/34619 (3.2%), risk difference -0.1% (95% CI -0.4% to 0.3%), p=0.96], AKI [2357/33897 (7%) vs 2391/33795 (7.1%), risk difference -0.1% (-0.6% to 0.4%), p=0.96], or delirium [1283/3928 (32.7%) vs 1279/3989 (32.1%), risk difference 0.6% (-2.0% to 3.2%), p=0.96]. There were no significant differences in secondary outcomes or in sensitivity analyses. Conclusions: In this large RCT of a novel application of telemedicine-based remote monitoring and decision support using real-time alerts and case reviews, we found no significant differences in postoperative outcomes. Large-scale intraoperative telemedicine is feasible, and we suggest future avenues where it may be impactful.

User-Centered Design of a Machine Learning Dashboard for Prediction of Postoperative Complications.

BACKGROUND: Machine learning models can help anesthesiology clinicians assess patients and make clinical and operational decisions, but well-designed human-computer interfaces are necessary for machine learning model predictions to result in clinician actions that help patients. Therefore, the goal of this study was to apply a user-centered design framework to create a user interface for displaying machine learning model predictions of postoperative complications to anesthesiology clinicians. METHODS: Twenty-five anesthesiology clinicians (attending anesthesiologists, resident physicians, and certified registered nurse anesthetists) participated in a 3-phase study that included (phase 1) semistructured focus group interviews and a card sorting activity to characterize user workflows and needs; (phase 2) simulated patient evaluation incorporating a low-fidelity static prototype display interface followed by a semistructured interview; and (phase 3) simulated patient evaluation with concurrent think-aloud incorporating a high-fidelity prototype display interface in the electronic health record. In each phase, data analysis included open coding of session transcripts and thematic analysis. RESULTS: During the needs assessment phase (phase 1), participants voiced that (a) identifying preventable risk related to modifiable risk factors is more important than nonpreventable risk, (b) comprehensive patient evaluation follows a systematic approach that relies heavily on the electronic health record, and (c) an easy-to-use display interface should have a simple layout that uses color and graphs to minimize time and energy spent reading it. When performing simulations using the low-fidelity prototype (phase 2), participants reported that (a) the machine learning predictions helped them to evaluate patient risk, (b) additional information about how to act on the risk estimate would be useful, and (c) correctable problems related to textual content existed. When performing simulations using the high-fidelity prototype (phase 3), usability problems predominantly related to the presentation of information and functionality. Despite the usability problems, participants rated the system highly on the System Usability Scale (mean score, 82.5; standard deviation, 10.5). CONCLUSIONS: Incorporating user needs and preferences into the design of a machine learning dashboard results in a display interface that clinicians rate as highly usable. Because the system demonstrates usability, evaluation of the effects of implementation on both process and clinical outcomes is warranted.

An Intraoperative Telemedicine Program to Improve Perioperative Quality Measures: The ACTFAST-3 Randomized Clinical Trial.

Importance: Telemedicine for clinical decision support has been adopted in many health care settings, but its utility in improving intraoperative care has not been assessed. Objective: To pilot the implementation of a real-time intraoperative telemedicine decision support program and evaluate whether it reduces postoperative hypothermia and hyperglycemia as well as other quality of care measures. Design, Setting, and Participants: This single-center pilot randomized clinical trial (Anesthesiology Control Tower-Feedback Alerts to Supplement Treatments [ACTFAST-3]) was conducted from April 3, 2017, to June 30, 2019, at a large academic medical center in the US. A total of 26 254 adult surgical patients were randomized to receive either usual intraoperative care (control group; n = 12 980) or usual care augmented by telemedicine decision support (intervention group; n = 13 274). Data were initially analyzed from April 22 to May 19, 2021, with updates in November 2022 and February 2023. Intervention: Patients received either usual care (medical direction from the anesthesia care team) or intraoperative anesthesia care monitored and augmented by decision support from the Anesthesiology Control Tower (ACT), a real-time, live telemedicine intervention. The ACT incorporated remote monitoring of operating rooms by a team of anesthesia clinicians with customized analysis software. The ACT reviewed alerts and electronic health record data to inform recommendations to operating room clinicians. Main Outcomes and Measures: The primary outcomes were avoidance of postoperative hypothermia (defined as the proportion of patients with a final recorded intraoperative core temperature >36 °C) and hyperglycemia (defined as the proportion of patients with diabetes who had a blood glucose level ≤180 mg/dL on arrival to the postanesthesia recovery area). Secondary outcomes included intraoperative hypotension, temperature monitoring, timely antibiotic redosing, intraoperative glucose evaluation and management, neuromuscular blockade documentation, ventilator management, and volatile anesthetic overuse. Results: Among 26 254 participants, 13 393 (51.0%) were female and 20 169 (76.8%) were White, with a median (IQR) age of 60 (47-69) years. There was no treatment effect on avoidance of hyperglycemia (7445 of 8676 patients [85.8%] in the intervention group vs 7559 of 8815 [85.8%] in the control group; rate ratio [RR], 1.00; 95% CI, 0.99-1.01) or hypothermia (7602 of 11 447 patients [66.4%] in the intervention group vs 7783 of 11 672 [66.7.%] in the control group; RR, 1.00; 95% CI, 0.97-1.02). Intraoperative glucose measurement was more common among patients with diabetes in the intervention group (RR, 1.07; 95% CI, 1.01-1.15), but other secondary outcomes were not significantly different. Conclusions and Relevance: In this randomized clinical trial, anesthesia care quality measures did not differ between groups, with high confidence in the findings. These results suggest that the intervention did not affect the targeted care practices. Further streamlining of clinical decision support and workflows may help the intraoperative telemedicine program achieve improvement in targeted clinical measures. Trial Registration: ClinicalTrials.gov Identifier: NCT02830126.

Protocol for the perioperative outcome risk assessment with computer learning enhancement (Periop ORACLE) randomized study.

Background: More than four million people die each year in the month following surgery, and many more experience complications such as acute kidney injury. Some of these outcomes may be prevented through early identification of at-risk patients and through intraoperative risk mitigation. Telemedicine has revolutionized the way at-risk patients are identified in critical care, but intraoperative telemedicine services are not widely used in anesthesiology. Clinicians in telemedicine settings may assist with risk stratification and brainstorm risk mitigation strategies while clinicians in the operating room are busy performing other patient care tasks. Machine learning tools may help clinicians in telemedicine settings leverage the abundant electronic health data available in the perioperative period. The primary hypothesis for this study is that anesthesiology clinicians can predict postoperative complications more accurately with machine learning assistance than without machine learning assistance. Methods: This investigation is a sub-study nested within the TECTONICS randomized clinical trial (NCT03923699). As part of TECTONICS, study team members who are anesthesiology clinicians working in a telemedicine setting are currently reviewing ongoing surgical cases and documenting how likely they feel the patient is to experience 30-day in-hospital death or acute kidney injury. For patients who are included in this sub-study, these case reviews will be randomized to be performed with access to a display showing machine learning predictions for the postoperative complications or without access to the display. The accuracy of the predictions will be compared across these two groups. Conclusion: Successful completion of this study will help define the role of machine learning not only for intraoperative telemedicine, but for other risk assessment tasks before, during, and after surgery. Registration: ORACLE is registered on ClinicalTrials.gov: NCT05042804; registered September 13, 2021.

Effect of electroencephalogram-guided anaesthesia administration on 1-yr mortality: follow-up of a randomised clinical trial.

BACKGROUND: Intraoperative EEG suppression duration has been associated with postoperative delirium and mortality. In a clinical trial testing anaesthesia titration to avoid EEG suppression, the intervention did not decrease the incidence of postoperative delirium, but was associated with reduced 30-day mortality. The present study evaluated whether the EEG-guided anaesthesia intervention was also associated with reduced 1-yr mortality. METHODS: This manuscript reports 1 yr follow-up of subjects from a single-centre RCT, including a post hoc secondary outcome (1-yr mortality) in addition to pre-specified secondary outcomes. The trial included subjects aged 60 yr or older undergoing surgery with general anaesthesia between January 2015 and May 2018. Patients were randomised to receive EEG-guided anaesthesia or usual care. The previously reported primary outcome was postoperative delirium. The outcome of the current study was all-cause 1-yr mortality. RESULTS: Of the 1232 subjects enrolled, 614 subjects were randomised to EEG-guided anaesthesia and 618 subjects to usual care. One-year mortality was 57/591 (9.6%) in the guided group and 62/601 (10.3%) in the usual-care group. No significant difference in mortality was observed (adjusted absolute risk difference, -0.7%; 99.5% confidence interval, -5.8% to 4.3%; P=0.68). CONCLUSIONS: An EEG-guided anaesthesia intervention aiming to decrease duration of EEG suppression during surgery did not significantly decrease 1-yr mortality. These findings, in the context of other studies, do not provide supportive evidence for EEG-guided anaesthesia to prevent intermediate term postoperative death. CLINICAL TRIAL REGISTRATION: NCT02241655.

Protocol for a proof-of-concept observational study evaluating the potential utility and acceptability of a telemedicine solution for the post-anesthesia care unit.

Introduction: The post-anesthesia care unit (PACU) is a clinical area designated for patients recovering from invasive procedures. There are typically several geographically dispersed PACUs within hospitals. Patients in the PACU can be unstable and at risk for complications. However, clinician coverage and patient monitoring in PACUs is not well regulated and might be sub-optimal. We hypothesize that a telemedicine center for the PACU can improve key PACU functions. Objectives: The objective of this study is to demonstrate the potential utility and acceptability of a telemedicine center to complement the key functions of the PACU. These include participation in hand-off activities to and from the PACU, detection of physiological derangements, identification of symptoms requiring treatment, recognition of situations requiring emergency medical intervention, and determination of patient readiness for PACU discharge. Methods and analysis: This will be a single center prospective before-and-after proof-of-concept study. Adults (18 years and older) undergoing elective surgery and recovering in two selected PACU bays will be enrolled. During the initial three-month observation phase, clinicians in the telemedicine center will not communicate with clinicians in the PACU, unless there is a specific patient safety concern. During the subsequent three-month interaction phase, clinicians in the telemedicine center will provide structured decision support to PACU clinicians. The primary outcome will be time to PACU discharge readiness determination in the two study phases. The attitudes of key stakeholders towards the telemedicine center will be assessed. Other outcomes will include detection of physiological derangements, complications, adverse symptoms requiring treatments, and emergencies requiring medical intervention. Registration: This trial is registered on clinicaltrials.gov, NCT04020887 (16 th July 2019).

Preoperative Cognitive Abnormality, Intraoperative Electroencephalogram Suppression, and Postoperative Delirium: A Mediation Analysis.

BACKGROUND: Postoperative delirium is a common complication that hinders recovery after surgery. Intraoperative electroencephalogram suppression has been linked to postoperative delirium, but it is unknown if this relationship is causal or if electroencephalogram suppression is merely a marker of underlying cognitive abnormalities. The hypothesis of this study was that intraoperative electroencephalogram suppression mediates a nonzero portion of the effect between preoperative abnormal cognition and postoperative delirium. METHODS: This is a prespecified secondary analysis of the Electroencephalography Guidance of Anesthesia to Alleviate Geriatric Syndromes (ENGAGES) randomized trial, which enrolled patients age 60 yr or older undergoing surgery with general anesthesia at a single academic medical center between January 2015 and May 2018. Patients were randomized to electroencephalogram-guided anesthesia or usual care. Preoperative abnormal cognition was defined as a composite of previous delirium, Short Blessed Test cognitive score greater than 4 points, or Eight Item Interview to Differentiate Aging and Dementia score greater than 1 point. Duration of intraoperative electroencephalogram suppression was defined as number of minutes with suppression ratio greater than 1%. Postoperative delirium was detected via Confusion Assessment Method or chart review on postoperative days 1 to 5. RESULTS: Among 1,113 patients, 430 patients showed evidence of preoperative abnormal cognition. These patients had an increased incidence of postoperative delirium (151 of 430 [35%] vs.123 of 683 [18%], P < 0.001). Of this 17.2% total effect size (99.5% CI, 9.3 to 25.1%), an absolute 2.4% (99.5% CI, 0.6 to 4.8%) was an indirect effect mediated by electroencephalogram suppression, while an absolute 14.8% (99.5% CI, 7.2 to 22.5%) was a direct effect of preoperative abnormal cognition. Randomization to electroencephalogram-guided anesthesia did not change the mediated effect size (P = 0.078 for moderation). CONCLUSIONS: A small portion of the total effect of preoperative abnormal cognition on postoperative delirium was mediated by electroencephalogram suppression. Study precision was too low to determine if the intervention changed the mediated effect.

Effect of Electroencephalography-Guided Anesthetic Administration on Postoperative Delirium Among Older Adults Undergoing Major Surgery: The ENGAGES Randomized Clinical Trial.

Importance: Intraoperative electroencephalogram (EEG) waveform suppression, often suggesting excessive general anesthesia, has been associated with postoperative delirium. Objective: To assess whether EEG-guided anesthetic administration decreases the incidence of postoperative delirium. Design, Setting, and Participants: Randomized clinical trial of 1232 adults aged 60 years and older undergoing major surgery and receiving general anesthesia at Barnes-Jewish Hospital in St Louis. Recruitment was from January 2015 to May 2018, with follow-up until July 2018. Interventions: Patients were randomized 1:1 (stratified by cardiac vs noncardiac surgery and positive vs negative recent fall history) to receive EEG-guided anesthetic administration (n = 614) or usual anesthetic care (n = 618). Main Outcomes and Measures: The primary outcome was incident delirium during postoperative days 1 through 5. Intraoperative measures included anesthetic concentration, EEG suppression, and hypotension. Adverse events included undesirable intraoperative movement, intraoperative awareness with recall, postoperative nausea and vomiting, medical complications, and death. Results: Of the 1232 randomized patients (median age, 69 years [range, 60 to 95]; 563 women [45.7%]), 1213 (98.5%) were assessed for the primary outcome. Delirium during postoperative days 1 to 5 occurred in 157 of 604 patients (26.0%) in the guided group and 140 of 609 patients (23.0%) in the usual care group (difference, 3.0% [95% CI, -2.0% to 8.0%]; P = .22). Median end-tidal volatile anesthetic concentration was significantly lower in the guided group than the usual care group (0.69 vs 0.80 minimum alveolar concentration; difference, -0.11 [95% CI, -0.13 to -0.10), and median cumulative time with EEG suppression was significantly less (7 vs 13 minutes; difference, -6.0 [95% CI, -9.9 to -2.1]). There was no significant difference between groups in the median cumulative time with mean arterial pressure below 60 mm Hg (7 vs 7 minutes; difference, 0.0 [95% CI, -1.7 to 1.7]). Undesirable movement occurred in 137 patients (22.3%) in the guided and 95 (15.4%) in the usual care group. No patients reported intraoperative awareness. Postoperative nausea and vomiting was reported in 48 patients (7.8%) in the guided and 55 patients (8.9%) in the usual care group. Serious adverse events were reported in 124 patients (20.2%) in the guided and 130 (21.0%) in the usual care group. Within 30 days of surgery, 4 patients (0.65%) in the guided group and 19 (3.07%) in the usual care group died. Conclusions and Relevance: Among older adults undergoing major surgery, EEG-guided anesthetic administration, compared with usual care, did not decrease the incidence of postoperative delirium. This finding does not support the use of EEG-guided anesthetic administration for this indication. Trial Registration: ClinicalTrials.gov Identifier: NCT02241655.

Protocol for the Effectiveness of an Anesthesiology Control Tower System in Improving Perioperative Quality Metrics and Clinical Outcomes: the TECTONICS randomized, pragmatic trial.

Introduction: Perioperative morbidity is a public health priority, and surgical volume is increasing rapidly. With advances in technology, there is an opportunity to research the utility of a telemedicine-based control center for anesthesia clinicians that assess risk, diagnoses negative patient trajectories, and implements evidence-based practices. Objectives: The primary objective of this trial is to determine whether an anesthesiology control tower (ACT) prevents clinically relevant adverse postoperative outcomes including 30-day mortality, delirium, respiratory failure, and acute kidney injury. Secondary objectives are to determine whether the ACT improves perioperative quality of care metrics including management of temperature, mean arterial pressure, mean airway pressure with mechanical ventilation, blood glucose, anesthetic concentration, antibiotic redosing, and efficient fresh gas flow. Methods and analysis: We are conducting a single center, randomized, controlled, phase 3 pragmatic clinical trial. A total of 58 operating rooms are randomized daily to receive support from the ACT or not. All adults (eighteen years and older) undergoing surgical procedures in these operating rooms are included and followed until 30 days after their surgery. Clinicians in operating rooms randomized to ACT support receive decision support from clinicians in the ACT. In operating rooms randomized to no intervention, the current standard of anesthesia care is delivered. The intention-to-treat principle will be followed for all analyses. Differences between groups will be presented with 99% confidence intervals; p-values <0.005 will be reported as providing compelling evidence, and p-values between 0.05 and 0.005 will be reported as providing suggestive evidence. Registration: TECTONICS is registered on ClinicalTrials.gov, NCT03923699; registered on 23 April 2019.

Using machine learning techniques to develop forecasting algorithms for postoperative complications: protocol for a retrospective study.

INTRODUCTION: Mortality and morbidity following surgery are pressing public health concerns in the USA. Traditional prediction models for postoperative adverse outcomes demonstrate good discrimination at the population level, but the ability to forecast an individual patient's trajectory in real time remains poor. We propose to apply machine learning techniques to perioperative time-series data to develop algorithms for predicting adverse perioperative outcomes. METHODS AND ANALYSIS: This study will include all adult patients who had surgery at our tertiary care hospital over a 4-year period. Patient history, laboratory values, minute-by-minute intraoperative vital signs and medications administered will be extracted from the electronic medical record. Outcomes will include in-hospital mortality, postoperative acute kidney injury and postoperative respiratory failure. Forecasting algorithms for each of these outcomes will be constructed using density-based logistic regression after employing a Nadaraya-Watson kernel density estimator. Time-series variables will be analysed using first and second-order feature extraction, shapelet methods and convolutional neural networks. The algorithms will be validated through measurement of precision and recall. ETHICS AND DISSEMINATION: This study has been approved by the Human Research Protection Office at Washington University in St Louis. The successful development of these forecasting algorithms will allow perioperative healthcare clinicians to predict more accurately an individual patient's risk for specific adverse perioperative outcomes in real time. Knowledge of a patient's dynamic risk profile may allow clinicians to make targeted changes in the care plan that will alter the patient's outcome trajectory. This hypothesis will be tested in a future randomised controlled trial.