Leveraging explainable artificial intelligence to optimize clinical decision support.

OBJECTIVE: To develop and evaluate a data-driven process to generate suggestions for improving alert criteria using explainable artificial intelligence (XAI) approaches. METHODS: We extracted data on alerts generated from January 1, 2019 to December 31, 2020, at Vanderbilt University Medical Center. We developed machine learning models to predict user responses to alerts. We applied XAI techniques to generate global explanations and local explanations. We evaluated the generated suggestions by comparing with alert's historical change logs and stakeholder interviews. Suggestions that either matched (or partially matched) changes already made to the alert or were considered clinically correct were classified as helpful. RESULTS: The final dataset included 2 991 823 firings with 2689 features. Among the 5 machine learning models, the LightGBM model achieved the highest Area under the ROC Curve: 0.919 [0.918, 0.920]. We identified 96 helpful suggestions. A total of 278 807 firings (9.3%) could have been eliminated. Some of the suggestions also revealed workflow and education issues. CONCLUSION: We developed a data-driven process to generate suggestions for improving alert criteria using XAI techniques. Our approach could identify improvements regarding clinical decision support (CDS) that might be overlooked or delayed in manual reviews. It also unveils a secondary purpose for the XAI: to improve quality by discovering scenarios where CDS alerts are not accepted due to workflow, education, or staffing issues.

A Prospective, Randomized, Open-Label Study Comparing an Opioid-Sparing Postsurgical Pain Management Protocol With and Without Liposomal Bupivacaine for Full-Arch Implant Surgery.

PURPOSE: Mild to moderate pain is common following dental implant surgeries, although severe pain is reported in some patients. This randomized, open-label trial was designed to determine the efficacy and safety of an opioid-sparing postsurgical pain management protocol with or without local infiltration of liposomal bupivacaine for full-arch implant surgery. This procedure is used in edentulous and/or failing dentition patients and involves securing four or more implants to the maxilla and/or mandible to serve as anchors for dental prostheses. MATERIALS AND METHODS: Patients scheduled to undergo full-arch implant surgery were randomly assigned to receive an opioid-sparing postsurgical pain management protocol with or without liposomal bupivacaine 266 mg at the end of surgery. All patients received infiltration with ≤ 40 mL lidocaine 2% with epinephrine at the beginning of surgery and bupivacaine 0.5% with epinephrine near the end of surgery and oral opioid or nonopioid analgesics (oxycodone 5 mg tablets or ibuprofen 600 mg), as needed, postsurgically. Pain severity at the surgical site was assessed using a verbal 0 to 10 numeric rating scale (0 [no pain] to 10 [worst pain imaginable]). Patients separately assessed pain in their mandible and maxilla. Reports of treatment-emergent adverse events were collected. RESULTS: Sixty-nine patients were randomized to the liposomal bupivacaine 266 mg (n = 34) or control group (n = 35). At all time points postsurgery for both the mandible (all P ≤ .0112) and the maxilla (all P ≤ .0083), the liposomal bupivacaine group reported significantly less cumulative pain than the control group. At the conclusion of the 7-day follow-up, patients in the liposomal bupivacaine group experienced one-third less cumulative postsurgical pain than patients in the control group. Seventy-seven percent of patients in the liposomal bupivacaine group and 80% in the control group experienced a treatment-emergent adverse event. A higher percentage of patients in the liposomal bupivacaine versus control group reported itching (15% vs 9%) and constipation (38% vs 23%). CONCLUSION: Patients receiving an opioid-sparing postsurgical pain management protocol with liposomal bupivacaine 266 mg experienced a statistically significant reduction of postsurgical pain and clinically relevant reduction in opioid consumption. The overall incidence of treatment-emergent adverse events was comparable in patients receiving an opioid-sparing postsurgical pain management protocol with or without liposomal bupivacaine.

Parenteral Protein Decision Support System Improves Protein Delivery in Preterm Infants: A Randomized Clinical Trial.

BACKGROUND: Management of neonatal parenteral protein intake for preterm infants is challenging and requires daily modifications of the dose to account for the infant's postnatal age, birth weight, current weight, and the volume and protein concentration of concurrent enteral nutrition. The objective of this study was to create and evaluate the Parenteral Protein Calculator (PPC), a clinical decision support system to improve the accuracy of protein intake for preterm infants who require parenteral nutrition (PN). MATERIALS AND METHODS: We integrated the PPC into the computerized provider order entry system and tested it in a randomized controlled trial (routine or PPC). Infants were eligible if they were ≤3 days old, had a birth weight ≤1500 g, and had no inborn error of metabolism. The primary outcome was the appropriate total protein intake, defined as target protein dose ±0.5 g/kg. RESULTS: We randomly allocated 42 infants for 221 PN days in the control group and 211 in the PPC group. Total protein intake in the PPC group was more accurate as compared with the control group (appropriate protein dosing: odds ratio = 5.8; 95% CI, 2.7-12.4). Absolute deviation from protein target was 0.41 g/kg (0.24-0.58) lower in the PPC group. CONCLUSION: The PPC improved appropriate protein dosing for premature infants receiving PN. Further studies are needed to test whether clinical decision support systems will reduce uremia and improve growth and to replicate similar findings in the cases of other PN nutrients.

Reducing Inappropriate Testing for the Evaluation of Diarrhea Among Hospitalized Patients.

BACKGROUND: Diarrhea is one of the most common illnesses in the United States. Evaluation frequently does not follow established guidelines. The objective of this study was to evaluate the effectiveness of a computerized physician order entry-based test guidance algorithm with regard to the clinical, financial, and operational impacts. METHODS: Our population was patients with diarrheal illness at a tertiary academic medical center. The intervention was a computerized physician order entry-based test guidance algorithm that restricted the use of stool cultures and ova and parasites testing of diarrhea in the adult inpatient location vs nonintervention sites, which were the emergency department, pediatric inpatient and adult and pediatric outpatient locations. We measured stool culture, ova and parasites, and Clostridium difficile testing rates from July 1, 2012 to January 31, 2016. Additionally, we calculated advisor usage, consults generated, accuracy of information, and cost savings. RESULTS: There was a significant decrease in stool culture and ova and parasites testing rates at the adult inpatient (P = .001 for both), pediatric (P < .001 for both), and adult emergency department (P < .001; P = .009) locations. The decrease at the intervention site was immediate, whereas the other locations showed a delayed but sustained decrease that suggests a collateral impact. A significant increase in the rate of stool culture and ova and parasites testing was observed in the outpatient setting (P = .02 and P = .001). We estimate that $21,931 was saved annually. CONCLUSIONS: A point-of-order test restriction algorithm for hospitalized adults with diarrhea reduced stool testing. Similar programs should be considered at other institutions and for the evaluation of other conditions.

Clinical decision support for atypical orders: detection and warning of atypical medication orders submitted to a computerized provider order entry system.

The specificity of medication-related alerts must be improved to overcome the pernicious effects of alert fatigue. A systematic comparison of new drug orders to historical orders could improve alert specificity and relevance. Using historical order data from a computerized provider order entry system, we alerted physicians to atypical orders during the prescribing of five medications: calcium, clopidogrel, heparin, magnesium, and potassium. The percentage of atypical orders placed for these five medications decreased during the 92 days the alerts were active when compared to the same period in the previous year (from 0.81% to 0.53%; p=0.015). Some atypical orders were appropriate. Fifty of the 68 atypical order alerts were over-ridden (74%). However, the over-ride rate is misleading because 28 of the atypical medication orders (41%) were changed. Atypical order alerts were relatively few, identified problems with frequencies as well as doses, and had a higher specificity than dose check alerts.