Clinical Knowledge and Reasoning Abilities of AI Large Language Models in Anesthesiology: A Comparative Study on the American Board of Anesthesiology Examination.

BACKGROUND: Over the past decade, artificial intelligence (AI) has expanded significantly with increased adoption across various industries, including medicine. Recently, AI-based large language models such as Generative Pretrained Transformer-3 (GPT-3), Bard, and Generative Pretrained Transformer-3 (GPT-4) have demonstrated remarkable language capabilities. While previous studies have explored their potential in general medical knowledge tasks, here we assess their clinical knowledge and reasoning abilities in a specialized medical context. METHODS: We studied and compared the performance of all 3 models on both the written and oral portions of the comprehensive and challenging American Board of Anesthesiology (ABA) examination, which evaluates candidates' knowledge and competence in anesthesia practice. RESULTS: Our results reveal that only GPT-4 successfully passed the written examination, achieving an accuracy of 78% on the basic section and 80% on the advanced section. In comparison, the less recent or smaller GPT-3 and Bard models scored 58% and 47% on the basic examination, and 50% and 46% on the advanced examination, respectively. Consequently, only GPT-4 was evaluated in the oral examination, with examiners concluding that it had a reasonable possibility of passing the structured oral examination. Additionally, we observe that these models exhibit varying degrees of proficiency across distinct topics, which could serve as an indicator of the relative quality of information contained in the corresponding training datasets. This may also act as a predictor for determining which anesthesiology subspecialty is most likely to witness the earliest integration with AI. CONCLUSIONS: GPT-4 outperformed GPT-3 and Bard on both basic and advanced sections of the written ABA examination, and actual board examiners considered GPT-4 to have a reasonable possibility of passing the real oral examination; these models also exhibit varying degrees of proficiency across distinct topics.

Clinical Knowledge and Reasoning Abilities of AI Large Language Models in Anesthesiology: A Comparative Study on the ABA Exam.

Over the past decade, Artificial Intelligence (AI) has expanded significantly with increased adoption across various industries, including medicine. Recently, AI's large language models such as GPT-3, Bard, and GPT-4 have demonstrated remarkable language capabilities. While previous studies have explored their potential in general medical knowledge tasks, here we assess their clinical knowledge and reasoning abilities in a specialized medical context. We study and compare their performances on both the written and oral portions of the comprehensive and challenging American Board of Anesthesiology (ABA) exam, which evaluates candidates' knowledge and competence in anesthesia practice. In addition, we invited two board examiners to evaluate AI's answers without disclosing to them the origin of those responses. Our results reveal that only GPT-4 successfully passed the written exam, achieving an accuracy of 78% on the basic section and 80% on the advanced section. In comparison, the less recent or smaller GPT-3 and Bard models scored 58% and 47% on the basic exam, and 50% and 46% on the advanced exam, respectively. Consequently, only GPT-4 was evaluated in the oral exam, with examiners concluding that it had a high likelihood of passing the actual ABA exam. Additionally, we observe that these models exhibit varying degrees of proficiency across distinct topics, which could serve as an indicator of the relative quality of information contained in the corresponding training datasets. This may also act as a predictor for determining which anesthesiology subspecialty is most likely to witness the earliest integration with AI.

A Delphi Process to Identify Relevant Outcomes That May Be Associated With a Predictive Analytic Tool to Detect Hemodynamic Deterioration in the Intensive Care Unit.

Background The critical care literature has seen an increase in the development and validation of tools using artificial intelligence for early detection of patient events or disease onset in the intensive care unit (ICU). The hemodynamic stability index (HSI) was found to have an AUC of 0.82 in predicting the need for hemodynamic intervention in the ICU. Future studies using this tool may benefit from targeting those outcomes that are more relevant to clinicians and most achievable. Methods A three-round Delphi study was conducted with a panel of 10 critical care physicians and three nurses in the United States to identify outcomes that may be most relevant and achievable with the HSI when evaluated for use in the ICU. To achieve criteria for relevance, at least 65% of panelists had to rate an outcome as a 4 or 5 on a 5-point scale. Results Nineteen of 24 outcomes that may be associated with the HSI achieved consensus for relevance. The Kemeny-Young approach was used to develop a matrix depicting the distribution of outcomes considering both relevance and achievability. "Reduces time spent in hemodynamic instability" and "reduces times to recognition of hemodynamic instability" were the highest-ranking outcomes considering both relevance and achievability. Conclusion This Delphi study was a feasible method to identify relevant outcomes that may be associated with an appropriate predictive analytic tool in the ICU. These findings can provide insight to researchers looking to study such tools to impact outcomes relevant to critical care practitioners. Future studies should test these tools in the ICU that target the most clinically relevant and achievable outcomes, such as time spent hemodynamically unstable or time until actionable nursing assessment or treatment.

Humanistic medicine in anaesthesiology: development and assessment of a curriculum in humanism for postgraduate anaesthesiology trainees.

BACKGROUND: An unintended consequence of medical technologies is loss of personal interactions and humanism between patients and their healthcare providers, leading to depersonalisation of medicine. As humanism is not integrated as part of formal postgraduate anaesthesiology education curricula, our goal was to design, introduce, and evaluate a comprehensive humanism curriculum into anaesthesiology training. METHODS: Subject-matter experts developed and delivered the humanism curriculum, which included interactive workshops, simulation sessions, formal feedback, and patient immersion experience. The effectiveness of the programme was evaluated using pre- and post-curriculum assessments in first-year postgraduate trainee doctors (residents). RESULTS: The anaesthesiology residents reported high satisfaction scores. Pre-/post-Jefferson Scale of Patient Perceptions of Physician Empathy showed an increase in empathy ratings with a median improvement of 12 points (range; P=0.013). After training, patients rated the residents as more empathetic (31 [4] vs 22 [5]; P<0.001; 95% confidence interval [CI]: 7-12) and professional (47 [3] vs 35 [8]; P<0.001; 95% CI: 9-16). Patient overall satisfaction with their anaesthesia provider improved after training (51 [6] vs 37 [10]; P<0.001; 95% CI: 10-18). Patients rated their anxiety lower in the post-training period compared with pretraining (1.8 [2.3] vs 3.6 [1.6]; P=0.001; 95% CI: 0.8-2.9). Patient-reported pain scores decreased after training (2.3 [2.5] vs 3.8 [2.1]; P=0.010; 95% CI: 0.4-2.8). CONCLUSIONS: Implementation of a humanism curriculum during postgraduate anaesthesiology training was well accepted, and can result in increased physician empathy and professionalism. This may improve patient pain, anxiety, and overall satisfaction with perioperative care.

A Prospective Observational Cohort Study of Calls for Help in a Tertiary Care Academic Operating Room Suite.

While significant literature exists on hospital-based "code calls," there is a lack of research on calls for help in the operating room (OR). The purpose of this study was to quantify the rate and nature of calls for help in the OR of a tertiary care hospital. For a 1-year period, all calls were recorded in the main OR at The University of California, Irvine Medical Center. The average rate of calls per 1000 anesthesia hours was 1.4 (95% CI, 1.1-1.8), corresponding to a rate of 5.0 (3.8-6.5) calls per 1000 cases. Airway (44%), cardiac (32%), and hemorrhagic (11%) emergencies were the most common etiologies. Thirty-day mortality approached 11% for patients who required a call for help in the OR.

View of U.S. spine surgeons regarding cost reduction measures.

BACKGROUND: This is a cross-sectional study. Our objective is to survey spine surgeons' views of responsibility to reduce healthcare costs, enthusiasm for cost reduction strategies, and agreement regarding roles in cost containment. The rising cost of healthcare has spurred debate about reducing expenditures. Previous studies have found that attitudes of anesthesiologists are predominantly in alignment with those of American physicians, but less is known about the views of spine surgeons. METHODS: After obtaining institutional approval, an electronic survey was disseminated to active members of AO Spine North America (AOSNA) via email. Respondents were asked eight questions about their age, gender, years in practice, practice facility, political views and opinions regarding management of healthcare costs. RESULTS: From 91 respondents, most were under the age of 60 years (87%), male (96%), and in practice for less than 30 years (91%), practiced at university hospitals (47%) and held politically conservative views (47%). Most responsibility was allocated to hospital and health systems, health insurance companies, pharmaceutical companies, and device manufacturers. Respondents were most enthusiastic about rooting out fraud and abuse and aware of their role in managing the cost of healthcare. Spine surgeons who were in practice for longer were more enthusiastic about reducing cost by reducing overall physician reimbursement via bundled payments, Medicare payment reduction, ending fee-for-service, penalizing surgeons for patient readmissions, and lowering compensation to individual spine surgeons. CONCLUSIONS: Spine surgeons allocated responsibility to reduce healthcare costs to healthcare systems, were most enthusiastic about eliminating wasteful spending, and were in agreement regarding their responsibility to control the costs of healthcare. Compared to US physicians of various specialties and anesthesiologists, spine surgeons assigned less responsibility to trials lawyers and expressed markedly less enthusiasm for limiting access to expensive treatments.

Computer simulated modeling of healthy and diseased right ventricular and pulmonary circulation.

We have previously developed a simulated cardiovascular physiology model for in-silico testing and validation of novel closed-loop controllers. To date, a detailed model of the right heart and pulmonary circulation was not needed, as previous controllers were not intended for use in patients with cardiac or pulmonary pathology. With new development of controllers for vasopressors, and looking forward, for combined vasopressor-fluid controllers, modeling of right-sided and pulmonary pathology is now relevant to further in-silico validation, so we aimed to expand our existing simulation platform to include these elements. Our hypothesis was that the completed platform could be tuned and stabilized such that the distributions of a randomized sample of simulated patients' baseline characteristics would be similar to reported population values. Our secondary outcomes were to further test the system in representing acute right heart failure and pulmonary artery hypertension. After development and tuning of the right-sided circulation, the model was validated against clinical data from multiple previously published articles. The model was considered 'tuned' when 100% of generated randomized patients converged to stability (steady, physiologically-plausible compartmental volumes, flows, and pressures) and 'valid' when the means for the model data in each health condition were contained within the standard deviations for the published data for the condition. A fully described right heart and pulmonary circulation model including non-linear pressure/volume relationships and pressure dependent flows was created over a 6-month span. The model was successfully tuned such that 100% of simulated patients converged into a steady state within 30 s. Simulation results in the healthy state for central venous volume (3350 ± 132 ml) pulmonary blood volume (405 ± 39 ml), pulmonary artery pressures (systolic 20.8 ± 4.1 mmHg and diastolic 9.4 ± 1.8 mmHg), left atrial pressure (4.6 ± 0.8 mmHg), PVR (1.0 ± 0.2 wood units), and CI (3.8 ± 0.5 l/min/m2) all met criteria for acceptance of the model, though the standard deviations of LAP and CI were somewhat narrower than published comparators. The simulation results for right ventricular infarction also fell within the published ranges: pulmonary blood volume (727 ± 102 ml), pulmonary arterial pressures (30 ± 4 mmHg systolic, 12 ± 2 mmHg diastolic), left atrial pressure (13 ± 2 mmHg), PVR (1.6 ± 0.3 wood units), and CI (2.0 ± 0.4 l/min/m2) all fell within one standard deviation of the reported population values and vice-versa. In the pulmonary hypertension model, pulmonary blood volume of 615 ± 90 ml, pulmonary arterial pressures of 80 ± 14 mmHg systolic, 36 ± 7 mmHg diastolic, and the left atrial pressure of 11 ± 2 mmHg all met criteria for acceptance. For CI, the simulated value of 2.8 ± 0.4 l/min/m2 once again had a narrower spread than most of the published data, but fell inside of the SD of all published data, and the PVR value of 7.5 ± 1.6 wood units fell in the middle of the four published studies. The right-ventricular and pulmonary circulation simulation appears to be a reasonable approximation of the right-sided circulation for healthy physiology as well as the pathologic conditions tested.

Two-Year Follow-up Survey: Views of US Anesthesiologists About Health Care Costs and Future Practice Roles.

Anesthesiologists' perspectives on US health care finance reform are increasingly germane to recent policy reforms. The aim of this follow-up survey was to examine how anesthesiologists' views of health care costs and future practice roles have changed since 2014. Six thousand randomly chosen active members of the American Society of Anesthesiologists were again surveyed and were also asked several new questions regarding specialties and perioperative management. Results showed an increase in self-reported understanding of the perioperative surgical home. Government, insurance companies, and pharmaceutical companies saw an increase in perceived "major responsibility" for cost reduction. Respondents vastly preferred that patient care under the perioperative surgical home be multidisciplinary.

Effects of Modification of Pain Protocol on Incidence of Post Operative Nausea and Vomiting.

BACKGROUND: A Perioperative Surgical Home (PSH) care model applies a standardized multidisciplinary approach to patient care using evidence-based medicine to modify and improve protocols. Analysis of patient outcome measures, such as postoperative nausea and vomiting (PONV), allows for refinement of existing protocols to improve patient care. We aim to compare the incidence of PONV in patients who underwent primary total joint arthroplasty before and after modification of our PSH pain protocol. METHODS: All total joint replacement PSH (TJR-PSH) patients who underwent primary THA (n=149) or TKA (n=212) in the study period were included. The modified protocol added a single dose of intravenous (IV) ketorolac given in the operating room and oxycodone immediate release orally instead of IV Hydromorphone in the Post Anesthesia Care Unit (PACU). The outcomes were (1) incidence of PONV and (2) average pain score in the PACU. We also examined the effect of primary anesthetic (spinal vs. GA) on these outcomes. The groups were compared using chi-square tests of proportions. RESULTS: The incidence of post-operative nausea in the PACU decreased significantly with the modified protocol (27.4% vs. 38.1%, p=0.0442). There was no difference in PONV based on choice of anesthetic or procedure. Average PACU pain scores did not differ significantly between the two protocols. CONCLUSION: Simple modifications to TJR-PSH multimodal pain management protocol, with decrease in IV narcotic use, resulted in a lower incidence of postoperative nausea, without compromising average PACU pain scores. This report demonstrates the need for continuous monitoring of PSH pathways and implementation of revisions as needed.

Joint arthroplasty Perioperative Surgical Home: Impact of patient characteristics on postoperative outcomes.

AIM: To determine the impact of different characteristics on postoperative outcomes for patients in a joint arthroplasty Perioperative Surgical Home (PSH) program. METHODS: A retrospective review was performed for patients enrolled in a joint arthroplasty PSH program who had undergone primary total hip arthroplasty (THA) and total knee arthroplasty (TKA). Patients were preoperatively stratified based on specific procedure performed, age, gender, body mass index (BMI), American Society of Anesthesiologists Physical Classification System (ASA) score, and Charleston Comorbidity Index (CCI) score. The primary outcome criterion was hospital length of stay (LOS). Secondary criteria including operative room (OR) duration, transfusion rate, Post-Anesthesia Care Unit (PACU) stay, readmission rate, post-operative complications, and discharge disposition. For each outcome, the predictor variables were entered into a generalized linear model with appropriate response and assessed for predictive relationship to the dependent variable. Significance level was set to 0.05. RESULTS: A total of 337 patients, 200 in the TKA cohort and 137 in the THA cohort, were eligible for the study. Nearly two-third of patients were female. Patient age averaged 64 years and preoperative BMI averaged 29 kg/m(2). The majority of patients were ASA score III and CCI score 0. After analysis, ASA score was the only variable predictive for LOS (P = 0.0011) and each increase in ASA score above 2 increased LOS by approximately 0.5 d. ASA score was also the only variable predictive for readmission rate (P = 0.0332). BMI was the only variable predictive for PACU duration (P = 0.0136). Specific procedure performed, age, gender, and CCI score were not predictive for any of the outcome criteria. OR duration, transfusion rate, post-operative complications or discharge disposition were not significantly associated with any of the predictor variables. CONCLUSION: The joint arthroplasty PSH model reduces postoperative outcome variability for patients with different preoperative characteristics and medical comorbidities.

Is There a Benefit for Liposomal Bupivacaine Compared to a Traditional Periarticular Injection in Total Knee Arthroplasty Patients With a History of Chronic Opioid Use?

BACKGROUND: Postoperative pain after total knee arthroplasty (TKA) poses a major challenge. It delays mobilization, increases opioid consumption and side effects, and lengthens hospitalization. This challenge multiplies when treating an opioid-dependent population. We examined whether a novel suspended release local anesthetic, liposomal bupivacaine (LB) would improve pain control and decrease opioid consumption after TKA compared to a standard periarticular injection in opioid-dependent patients. METHODS: Thirty-eight patients undergoing TKA were randomly assigned to receive either a periarticular injection (PAI) with LB (n = 20) or with a standard PAI (including a combination of ropivacaine, clonidine, Toradol, Epinepherine, and saline; n = 18) as part of a multimodal pain management approach. All periarticular injections were done by a single surgeon. Perioperative treatment was similar between groups. Postoperative information regarding pain level was evaluated by a pain visual analog scale score. Postoperative opioid consumption was recorded. RESULTS: After controlling baseline narcotic usage before surgery, no differences were found between groups in daily postoperative narcotic usage (P = .113), average daily pain score (P = .332), or maximum daily pain score (P = .881). However, when examining pain levels separately for each day, pain visual analog scale scores were reported higher in post operative day 1 in the LB group (P = .033). CONCLUSIONS: LB was not found to be superior to standard PAI in opioid-dependent patients undergoing TKA. This patient population continues to present a challenge even with modern multimodal pain protocols.

Perioperative blood ordering optimization process using information from an anesthesia information management system.

BACKGROUND: As part of ongoing perioperative surgical home implantation process, we applied a previously published algorithm for creation of a maximum surgical blood order schedule (MSBOS) to our operating rooms. We hypothesized that using the MSBOS we could show a reduction in unnecessary preoperative blood testing and associated costs. STUDY DESIGN AND METHODS: Data regarding all surgical cases done at UC Irvine Health's operating rooms from January 1, 2011, to January 1, 2014 were extracted from the anesthesia information management systems (AIMS). After the data were organized into surgical specialties and operative sites, blood order recommendations were generated based on five specific case characteristics of the group. Next, we assessed current ordering practices in comparison to actual blood utilization to identify potential areas of wastage and performed a cost analysis comparing the annual hospital costs from preoperative blood orders if the blood order schedule were to be followed to historical practices. RESULTS: Of the 19,138 patients who were categorized by the MSBOS as needing no blood sample, 2694 (14.0%) had a type and screen (T/S) ordered and 1116 (5.8%) had a type and crossmatch ordered. Of the 6073 procedures where MSBOS recommended only a T/S, 2355 (38.8%) had blood crossmatched. The cost analysis demonstrated an annual reduction in actual hospital costs of $57,335 with the MSBOS compared to historical blood ordering practices. CONCLUSION: We showed that the algorithm for development of a multispecialty blood order schedule is transferable and yielded reductions in preoperative blood product screening at our institution.

Can tranexamic acid change preoperative anemia management during total joint arthroplasty?

AIM: To investigate the postoperative transfusion and complication rates of anemic and nonanemic total joint arthroplasty patients given tranexamic acid (TXA). METHODS: A cross-sectional prospective study was conducted of primary hip and knee arthroplasty cases performed from 11/2012 to 6/2014. Exclusion criteria included revision arthroplasty, bilateral arthroplasty, acute arthroplasty after fracture, and contraindication to TXA. Patients were screened prior to surgery, with anemia was defined as hemoglobin of less than 12 g/dL for females and of less than 13 g/dL for males. Patients were divided into four different groups, based on the type of arthroplasty (total hip or total knee) and hemoglobin status (anemic or nonanemic). Intraoperatively, all patients received 2 g of intravenous TXA during surgery. Postoperatively, allogeneic blood transfusion (ABT) was directed by both clinical symptoms and relative hemoglobin change. Complications were recorded within the first two weeks after surgery and included thromboembolism, infection, and wound breakdown. The differences in transfusion and complication rates, as well as the relative hemoglobin change, were compared between anemic and nonanemic groups. RESULTS: A total of 232 patients undergoing primary joint arthroplasty were included in the study. For the total hip arthroplasty cohort, 21% (18/84) of patients presented with preoperative anemia. Two patients in the anemic group and two patients in the nonanemic group needed ABTs; this was not significantly different (P = 0.20). One patient in the anemic group presented with a deep venous thromboembolism while no patients in the nonanemic group had an acute complication; this was not significantly different (P = 0.21). For nonanemic patients, the average change in hemoglobin was 2.73 ± 1.17 g/dL. For anemic patients, the average change in hemoglobin was 2.28 ± 0.96 g/dL. Between the two groups, the hemoglobin difference of 0.45 g/dL was not significant (P = 0.13). For the total knee arthroplasty cohort, 18% (26/148) of patients presented with preoperative anemia. No patients in either group required a blood transfusion or had an acute postoperative complication. For nonanemic patients, the average change in hemoglobin was 1.85 ± 0.79 g/dL. For anemic patients, the average change in hemoglobin was 1.09 ± 0.58 g/dL. Between the two groups, the hemoglobin difference of 0.76 g/dL was significant (P < 0.001). CONCLUSION: TXA administration results in low transfusion and complication rates and may be a useful adjunct for TJA patients with preoperative anemia.

Postoperative global amnesia reversed with flumazenil.

Global postoperative amnesia (profound anterograde and retrograde amnesia) is rare and usually attributed to transient global amnesia-a poorly understood condition with no broadly accepted mechanism. We report an incident of probable transient global amnesia in a patient after endoscopic retrograde cholangiopancreatogram under general anesthesia, which was successfully treated with flumazenil. On the basis of the results of flumazenil administration in this and a previous case report, we would recommend a trial dose of 0.2 mg for cases of global postoperative amnesia, repeated if the first dose seems effective.

Children's desire for perioperative information.

BACKGROUND: The purpose of this investigation was to identify what perioperative information children want to receive from the medical staff. METHODS: As a first step, we developed an instrument based on a qualitative study conducted with children in Great Britain, input from a focus group, and input from school children. On the day of surgery, 143 children aged 7-17 yr completed a 40-item assessment of desired surgical information and a measure of anxiety (State-Trait Anxiety Inventory for Children). Parents completed a measure assessing their child's temperament (Emotionality, Activity, Sociability, and Impulsivity Survey) and a measure of their own anxiety (State-Trait Anxiety Inventory). RESULTS: Results indicated that the vast majority of children had a desire for comprehensive information about their surgery, including information about pain and anesthesia, and procedural information and information about potential complications. The most highly endorsed items by children involved information about pain. Children who were more anxious endorsed a stronger desire for pain information and lesser tendency to avoid information. Younger children wanted to know what the perioperative environment would look like more than adolescent children. CONCLUSIONS: We conclude that the majority of children aged 7-17 yr who undergo surgery want to be given comprehensive perioperative information and health care providers should ensure adequate information regarding postoperative pain is provided.