Congenital Heart Surgery Machine Learning-Derived In-Depth Benchmarking Tool.

BACKGROUND: We previously showed that machine learning-based methodologies of optimal classification trees (OCTs) can accurately predict risk after congenital heart surgery and assess case-mix-adjusted performance after benchmark procedures. We extend this methodology to provide interpretable, easily accessible, and actionable hospital performance analysis across all procedures. METHODS: The European Congenital Heart Surgeons Association Congenital Cardiac Database data subset of 172,888 congenital cardiac surgical procedures performed in European centers between 1989 and 2022 was analyzed. OCT models (decision trees) were built predicting hospital mortality (area under the curve [AUC], 0.866), prolonged postoperative mechanical ventilatory support time (AUC, 0.851), or hospital length of stay (AUC, 0.818), thereby establishing case-adjusted benchmarking standards reflecting the overall performance of all participating hospitals, designated as the "virtual hospital." OCT analysis of virtual hospital aggregate data yielded predicted expected outcomes (both aggregate and for risk-matched patient cohorts) for the individual hospital's own specific case-mix, readily available on-line. RESULTS: Raw average rates were hospital mortality, 4.9%; mechanical ventilatory support time, 14.5%; and length of stay, 15.0%. Of 146 participating centers, compared with each hospital's overall case-adjusted predicted hospital mortality benchmark, 20.5% statistically (<90% CI) overperformed and 20.5% underperformed. An interactive tool based on the OCT analysis automatically reveals 14 hospital-specific patient cohorts, simultaneously assessing overperformance or underperformance, and enabling further analysis of cohort strata in any chosen time frame. CONCLUSIONS: Machine learning-based OCT benchmarking analysis provides automatic assessment of hospital-specific case-adjusted performance after congenital heart surgery, not only overall but importantly, also by similar risk patient cohorts. This is a tool for hospital self-assessment, particularly facilitated by the user-accessible online-platform.

Validation of the Al-based Predictive OpTimal Trees in Emergency Surgery Risk (POTTER) Calculator in Patients 65 Years and Older.

OBJECTIVE: We sought to assess the performance of the Predictive OpTimal Trees in Emergency Surgery Risk (POTTER) tool in elderly emergency surgery (ES) patients. SUMMARY BACKGROUND DATA: The POTTER tool was derived using a novel Artificial Intelligence (AI)-methodology called optimal classification trees and validated for prediction of ES outcomes. POTTER outperforms all existent risk-prediction models and is available as an interactive smartphone application. Predicting outcomes in elderly patients has been historically challenging and POTTER has not yet been tested in this population. METHODS: All patients ≥65 years who underwent ES in the ACS-NSQIP 2017 database were included. POTTER's performance for 30-day mortality and 18 postoperative complications (eg, respiratory or renal failure) was assessed using c-statistic methodology, with planned sub-analyses for patients 65 to 74, 75 to 84, and 85+ years. RESULTS: A total of 29,366 patients were included, with mean age 77, 55.8% females, and 62% who underwent emergency general surgery. POTTER predicted mortality accurately in all patients over 65 (c-statistic 0.80). Its best performance was in patients 65 to 74 years (c-statistic 0.84), and its worst in patients ≥85 years (c-statistic 0.71). POTTER had the best discrimination for predicting septic shock (c-statistic 0.90), respiratory failure requiring mechanical ventilation for ≥48 hours (c-statistic 0.86), and acute renal failure (c-statistic 0.85). CONCLUSIONS: POTTER is a novel, interpretable, and highly accurate predictor of in-hospital mortality in elderly ES patients up to age 85 years. POTTER could prove useful for bedside counseling and for benchmarking of ES care.

Validation of the artificial intelligence-based trauma outcomes predictor (TOP) in patients 65 years and older.

BACKGROUND: The Trauma Outcomes Predictor tool was recently derived using a machine learning methodology called optimal classification trees and validated for prediction of outcomes in trauma patients. The Trauma Outcomes Predictor is available as an interactive smartphone application. In this study, we sought to assess the performance of the Trauma Outcomes Predictor in the elderly trauma patient. METHODS: All patients aged 65 years and older in the American College of Surgeons-Trauma Quality Improvement Program 2017 database were included. The performance of the Trauma Outcomes Predictor in predicting in-hospital mortality and combined and specific morbidity based on incidence of 9 specific in-hospital complications was assessed using the c-statistic methodology, with planned subanalyses for patients 65 to 74, 75 to 84, and 85+ years. RESULTS: A total of 260,505 patients were included. Median age was 77 (71-84) years, 57% were women, and 98.8% had a blunt mechanism of injury. The Trauma Outcomes Predictor accurately predicted mortality in all patients, with excellent performance for penetrating trauma (c-statistic: 0.92) and good performance for blunt trauma (c-statistic: 0.83). Its best performance was in patients 65 to 74 years (c-statistic: blunt 0.86, penetrating 0.93). Among blunt trauma patients, the Trauma Outcomes Predictor had the best discrimination for predicting acute respiratory distress syndrome (c-statistic 0.75) and cardiac arrest requiring cardiopulmonary resuscitation (c-statistic 0.75). Among penetrating trauma patients, the Trauma Outcomes Predictor had the best discrimination for deep and organ space surgical site infections (c-statistics 0.95 and 0.84, respectively). CONCLUSION: The Trauma Outcomes Predictor is a novel, interpretable, and highly accurate predictor of in-hospital mortality in the elderly trauma patient up to age 85 years. The Trauma Outcomes Predictor could prove useful for bedside counseling of elderly patients and their families and for benchmarking the quality of geriatric trauma care.

Benchmarking cesarean delivery rates using machine learning-derived optimal classification trees.

OBJECTIVE: To establish a case-adjusted hospital-specific performance evaluation tool using machine learning methodology for cesarean delivery. DATA SOURCES: Secondary data were collected from patients between January 1, 2015 and February 28, 2018 using a hospital's "Electronic Data Warehouse" database from Illinois, USA. STUDY DESIGN: The machine learning methodology of optimal classification trees (OCTs) was used to predict cesarean delivery rate by physician group, thereby establishing the case-adjusted benchmarking standards in comparison to the overall hospital cesarean delivery rate. Outcomes of specific patient populations of each participating practice were predicted, as if each were treated in the overall hospital environment. The resulting OCTs estimate physician group expected cesarean delivery outcomes, both aggregate and in specific clinical situations. DATA COLLECTION/EXTRACTION METHODS: Twelve thousand eight hunderd and forty one singleton, vertex, term deliveries, cared for by practices with ≥50 births. PRINCIPAL FINDINGS: The overall rate of cesarean delivery was 18.6% (n = 2384), with a range of 13.3%-33.7% amongst 22 physician practices. An optimal decision tree was used to create a prediction model for the hospital overall, which defined 23 patient cohorts divided by 46 nodes. The model's performance for prediction of cesarean delivery is as follows: area under the curve 0.73, sensitivity 98.4%, specificity 16.1%, positive predictive value 83.7%, negative predictive value 70.6%. Comparisons with the overall hospital's specific-case adjusted benchmark groups revealed that several groups outperformed the overall hospital benchmark, and some practice groups underperformed in comparison to the overall hospital benchmark. CONCLUSIONS: OCT benchmarking can assess physician practice-specific case-adjusted performance, both overall and clinical situation-specific, and can serve as a valuable tool for hospital self-assessment and quality improvement.

Benchmarking in Congenital Heart Surgery Using Machine Learning-Derived Optimal Classification Trees.

Background: We have previously shown that the machine learning methodology of optimal classification trees (OCTs) can accurately predict risk after congenital heart surgery (CHS). We have now applied this methodology to define benchmarking standards after CHS, permitting case-adjusted hospital-specific performance evaluation. Methods: The European Congenital Heart Surgeons Association Congenital Database data subset (31 792 patients) who had undergone any of the 10 "benchmark procedure group" primary procedures were analyzed. OCT models were built predicting hospital mortality (HM), and prolonged postoperative mechanical ventilatory support time (MVST) or length of hospital stay (LOS), thereby establishing case-adjusted benchmarking standards reflecting the overall performance of all participating hospitals, designated as the "virtual hospital." These models were then used to predict individual hospitals' expected outcomes (both aggregate and, importantly, for risk-matched patient cohorts) for their own specific cases and case-mix, based on OCT analysis of aggregate data from the "virtual hospital." Results: The raw average rates were HM = 4.4%, MVST = 15.3%, and LOS = 15.5%. Of 64 participating centers, in comparison with each hospital's specific case-adjusted benchmark, 17.0% statistically (under 90% confidence intervals) overperformed and 26.4% underperformed with respect to the predicted outcomes for their own specific cases and case-mix. For MVST and LOS, overperformers were 34.0% and 26.4%, and underperformers were 28.3% and 43.4%, respectively. OCT analyses reveal hospital-specific patient cohorts of either overperformance or underperformance. Conclusions: OCT benchmarking analysis can assess hospital-specific case-adjusted performance after CHS, both overall and patient cohort-specific, serving as a tool for hospital self-assessment and quality improvement.

Adverse Outcomes Prediction for Congenital Heart Surgery: A Machine Learning Approach.

OBJECTIVE: Risk assessment tools typically used in congenital heart surgery (CHS) assume that various possible risk factors interact in a linear and additive fashion, an assumption that may not reflect reality. Using artificial intelligence techniques, we sought to develop nonlinear models for predicting outcomes in CHS. METHODS: We built machine learning (ML) models to predict mortality, postoperative mechanical ventilatory support time (MVST), and hospital length of stay (LOS) for patients who underwent CHS, based on data of more than 235,000 patients and 295,000 operations provided by the European Congenital Heart Surgeons Association Congenital Database. We used optimal classification trees (OCTs) methodology for its interpretability and accuracy, and compared to logistic regression and state-of-the-art ML methods (Random Forests, Gradient Boosting), reporting their area under the curve (AUC or c-statistic) for both training and testing data sets. RESULTS: Optimal classification trees achieve outstanding performance across all three models (mortality AUC = 0.86, prolonged MVST AUC = 0.85, prolonged LOS AUC = 0.82), while being intuitively interpretable. The most significant predictors of mortality are procedure, age, and weight, followed by days since previous admission and any general preoperative patient risk factors. CONCLUSIONS: The nonlinear ML-based models of OCTs are intuitively interpretable and provide superior predictive power. The associated risk calculator allows easy, accurate, and understandable estimation of individual patient risks, in the theoretical framework of the average performance of all centers represented in the database. This methodology has the potential to facilitate decision-making and resource optimization in CHS, enabling total quality management and precise benchmarking initiatives.

Trauma outcome predictor: An artificial intelligence interactive smartphone tool to predict outcomes in trauma patients.

BACKGROUND: Classic risk assessment tools often treat patients' risk factors as linear and additive. Clinical reality suggests that the presence of certain risk factors can alter the impact of other factors; in other words, risk modeling is not linear. We aimed to use artificial intelligence (AI) technology to design and validate a nonlinear risk calculator for trauma patients. METHODS: A novel, interpretable AI technology called Optimal Classification Trees (OCTs) was used in an 80:20 derivation/validation split of the 2010 to 2016 American College of Surgeons Trauma Quality Improvement Program database. Demographics, emergency department vital signs, comorbidities, and injury characteristics (e.g., severity, mechanism) of all blunt and penetrating trauma patients 18 years or older were used to develop, train then validate OCT algorithms to predict in-hospital mortality and complications (e.g., acute kidney injury, acute respiratory distress syndrome, deep vein thrombosis, pulmonary embolism, sepsis). A smartphone application was created as the algorithm's interactive and user-friendly interface. Performance was measured using the c-statistic methodology. RESULTS: A total of 934,053 patients were included (747,249 derivation; 186,804 validation). The median age was 51 years, 37% were women, 90.5% had blunt trauma, and the median Injury Severity Score was 11. Comprehensive OCT algorithms were developed for blunt and penetrating trauma, and the interactive smartphone application, Trauma Outcome Predictor (TOP) was created, where the answer to one question unfolds the subsequent one. Trauma Outcome Predictor accurately predicted mortality in penetrating injury (c-statistics: 0.95 derivation, 0.94 validation) and blunt injury (c-statistics: 0.89 derivation, 0.88 validation). The validation c-statistics for predicting complications ranged between 0.69 and 0.84. CONCLUSION: We suggest TOP as an AI-based, interpretable, accurate, and nonlinear risk calculator for predicting outcome in trauma patients. Trauma Outcome Predictor can prove useful for bedside counseling of critically injured trauma patients and their families, and for benchmarking the quality of trauma care.

Validation of the Artificial Intelligence-Based Predictive Optimal Trees in Emergency Surgery Risk (POTTER) Calculator in Emergency General Surgery and Emergency Laparotomy Patients.

BACKGROUND: The Predictive Optimal Trees in Emergency Surgery Risk (POTTER) tool is an artificial intelligence-based calculator for the prediction of 30-day outcomes in patients undergoing emergency operations. In this study, we sought to assess the performance of POTTER in the emergency general surgery (EGS) population in particular. METHODS: All patients who underwent EGS in the 2017 American College of Surgeons NSQIP database were included. The performance of POTTER in predicting 30-day postoperative mortality, morbidity, and 18 specific complications was assessed using the c-statistic metric. As a subgroup analysis, the performance of POTTER in predicting the outcomes of patients undergoing emergency laparotomy was assessed. RESULTS: A total of 59,955 patients were included. Median age was 50 years and 51.3% were women. POTTER predicted mortality (c-statistic = 0.93) and morbidity (c-statistic = 0.83) extremely well. Among individual complications, POTTER had the highest performance in predicting septic shock (c-statistic = 0.93), respiratory failure requiring mechanical ventilation for 48 hours or longer (c-statistic = 0.92), and acute renal failure (c-statistic = 0.92). Among patients undergoing emergency laparotomy, the c-statistic performances of POTTER in predicting mortality and morbidity were 0.86 and 0.77, respectively. CONCLUSIONS: POTTER is an interpretable, accurate, and user-friendly predictor of 30-day outcomes in patients undergoing EGS. POTTER could prove useful for bedside counseling of patients and their families and for benchmarking of EGS care.

Bayesian additive decision trees of biomarker by treatment interactions for predictive biomarker detection and subgroup identification.

Personalized medicine, or tailored therapy, has been an active and important topic in recent medical research. Many methods have been proposed in the literature for predictive biomarker detection and subgroup identification. In this article, we propose a novel decision tree-based approach applicable in randomized clinical trials. We model the prognostic effects of the biomarkers using additive regression trees and the biomarker-by-treatment effect using a single regression tree. Bayesian approach is utilized to periodically revise the split variables and the split rules of the decision trees, which provides a better overall fitting. Gibbs sampler is implemented in the MCMC procedure, which updates the prognostic trees and the interaction tree separately. We use the posterior distribution of the interaction tree to construct the predictive scores of the biomarkers and to identify the subgroup where the treatment is superior to the control. Numerical simulations show that our proposed method performs well under various settings comparing to existing methods. We also demonstrate an application of our method in a real clinical trial.

Safety and efficacy of insulin glargine 300 u/mL compared with other basal insulin therapies in patients with type 2 diabetes mellitus: a network meta-analysis.

OBJECTIVE: To compare the efficacy and safety of a concentrated formulation of insulin glargine (Gla-300) with other basal insulin therapies in patients with type 2 diabetes mellitus (T2DM). DESIGN: This was a network meta-analysis (NMA) of randomised clinical trials of basal insulin therapy in T2DM identified via a systematic literature review of Cochrane library databases, MEDLINE and MEDLINE In-Process, EMBASE and PsycINFO. OUTCOME MEASURES: Changes in HbA1c (%) and body weight, and rates of nocturnal and documented symptomatic hypoglycaemia were assessed. RESULTS: 41 studies were included; 25 studies comprised the main analysis population: patients on basal insulin-supported oral therapy (BOT). Change in glycated haemoglobin (HbA1c) was comparable between Gla-300 and detemir (difference: -0.08; 95% credible interval (CrI): -0.40 to 0.24), neutral protamine Hagedorn (NPH; 0.01; -0.28 to 0.32), degludec (-0.12; -0.42 to 0.20) and premixed insulin (0.26; -0.04 to 0.58). Change in body weight was comparable between Gla-300 and detemir (0.69; -0.31 to 1.71), NPH (-0.76; -1.75 to 0.21) and degludec (-0.63; -1.63 to 0.35), but significantly lower compared with premixed insulin (-1.83; -2.85 to -0.75). Gla-300 was associated with a significantly lower nocturnal hypoglycaemia rate versus NPH (risk ratio: 0.18; 95% CrI: 0.05 to 0.55) and premixed insulin (0.36; 0.14 to 0.94); no significant differences were noted in Gla-300 versus detemir (0.52; 0.19 to 1.36) and degludec (0.66; 0.28 to 1.50). Differences in documented symptomatic hypoglycaemia rates of Gla-300 versus detemir (0.63; 0.19 to 2.00), NPH (0.66; 0.27 to 1.49) and degludec (0.55; 0.23 to 1.34) were not significant. Extensive sensitivity analyses supported the robustness of these findings. CONCLUSIONS: NMA comparisons are useful in the absence of direct randomised controlled data. This NMA suggests that Gla-300 is also associated with a significantly lower risk of nocturnal hypoglycaemia compared with NPH and premixed insulin, with glycaemic control comparable to available basal insulin comparators.

Comparative Effectiveness of Second-Line Targeted Therapies for Metastatic Renal Cell Carcinoma: A Systematic Review and Meta-Analysis of Real-World Observational Studies.

OBJECTIVE: The optimal sequencing of targeted therapies for metastatic renal cell carcinoma (mRCC) is unknown. Observational studies with a variety of designs have reported differing results. The objective of this study is to systematically summarize and interpret the published real-world evidence comparing sequential treatment for mRCC. METHODS: A search was conducted in Medline and Embase (2009-2013), and conference proceedings from American Society of Clinical Oncology (ASCO), ASCO Genitourinary Cancers Symposium (ASCO-GU), and European Society for Medical Oncology (ESMO) (2011-2013). We systematically reviewed observational studies comparing second-line mRCC treatment with mammalian target of rapamycin inhibitors (mTORi) versus vascular endothelial growth factor (VEGF) tyrosine kinase inhibitors (TKI). Studies were evaluated for 1) use of a retrospective cohort design after initiation of second-line therapy, 2) adjustment for patient characteristics, and 3) use of data from multiple centers. Meta-analyses were conducted for comparisons of overall survival (OS) and progression-free survival (PFS). RESULTS: Ten studies reported OS and exhibited significant heterogeneity in estimated second-line treatment effects (I2 = 68%; P = 0.001). Four of these were adjusted, multicenter, retrospective cohort studies, and these showed no evidence of heterogeneity (I2 = 0%; P = 0.61) and a significant association between second-line mTORi (>75% everolimus) and longer OS compared to VEGF TKI (>60% sorafenib) (HR = 0.82, 95% CI: 0.68 to 0.98) in a meta-analysis. Seven studies comparing PFS showed significant heterogeneity overall and among the adjusted, multicenter, retrospective cohort studies. Real-world observational data for axitinib outcomes was limited at the time of this study. CONCLUSIONS: Real-world studies employed different designs and reported heterogeneous results comparing the effectiveness of second-line mTORi and VEGF TKI in the treatment of mRCC. Within the subset of adjusted, multicenter observational studies, second-line use of mTORi was associated with significantly prolonged survival compared with second-line use of VEGF TKI.

A cost-analysis model for anticoagulant treatment in the hospital setting.

BACKGROUND: Rivaroxaban is the first oral factor Xa inhibitor approved in the US to reduce the risk of stroke and blood clots among people with non-valvular atrial fibrillation, treat deep vein thrombosis (DVT), treat pulmonary embolism (PE), reduce the risk of recurrence of DVT and PE, and prevent DVT and PE after knee or hip replacement surgery. The objective of this study was to evaluate the costs from a hospital perspective of treating patients with rivaroxaban vs other anticoagulant agents across these five populations. METHODS: An economic model was developed using treatment regimens from the ROCKET-AF, EINSTEIN-DVT and PE, and RECORD1-3 randomized clinical trials. The distribution of hospital admissions used in the model across the different populations was derived from the 2010 Healthcare Cost and Utilization Project database. The model compared total costs of anticoagulant treatment, monitoring, inpatient stay, and administration for patients receiving rivaroxaban vs other anticoagulant agents. The length of inpatient stay (LOS) was determined from the literature. RESULTS: Across all populations, rivaroxaban was associated with an overall mean cost savings of $1520 per patient. The largest cost savings associated with rivaroxaban was observed in patients with DVT or PE ($6205 and $2742 per patient, respectively). The main driver of the cost savings resulted from the reduction in LOS associated with rivaroxaban, contributing to ∼90% of the total savings. Furthermore, the overall mean anticoagulant treatment cost was lower for rivaroxaban vs the reference groups. LIMITATIONS: The distribution of patients across indications used in the model may not be generalizable to all hospitals, where practice patterns may vary, and average LOS cost may not reflect the actual reimbursements that hospitals received. CONCLUSION: From a hospital perspective, the use of rivaroxaban may be associated with cost savings when compared to other anticoagulant treatments due to lower drug cost and shorter LOS associated with rivaroxaban.

Cost-effectiveness of rivaroxaban compared with enoxaparin plus a vitamin K antagonist for the treatment of venous thromboembolism.

BACKGROUND: Venous thromboembolism (VTE), comprised of deep vein thrombosis (DVT) and pulmonary embolism (PE), is commonly treated with a low-molecular-weight heparin such as enoxaparin plus a vitamin K antagonist (VKA) to prevent recurrence. Administration of enoxaparin + VKA is hampered by complexities of laboratory monitoring and frequent dose adjustments. Rivaroxaban, an orally administered anticoagulant, has been compared with enoxaparin + VKA in the EINSTEIN trials. The objective was to evaluate the cost-effectiveness of rivaroxaban compared with enoxaparin + VKA as anticoagulation treatment for acute, symptomatic, objectively-confirmed DVT or PE. METHODS: A Markov model was built to evaluate the costs, quality-adjusted life-years (QALYs), and incremental cost-effectiveness ratios associated with rivaroxaban compared to enoxaparin + VKA in adult patients treated for acute DVT or PE. All patients entered the model in the 'on-treatment' state upon commencement of oral rivaroxaban or enoxaparin + VKA for 3, 6, or 12 months. Transition probabilities were obtained from the EINSTEIN trials during treatment and published literature after treatment. A 3-month cycle length, US payer perspective ($2012), 5-year time horizon and a 3% annual discount rate were used. RESULTS: Treatment with rivaroxaban cost $2,448 per-patient less and was associated with 0.0058 more QALYs compared with enoxaparin + VKA, making it a dominant economic strategy. Upon one-way sensitivity analysis, the model's results were sensitive to the reduction in index VTE hospitalization length-of-stay associated with rivaroxaban compared with enoxaparin + VKA. At a willingness-to-pay threshold of $50,000/QALY, probabilistic sensitivity analysis showed rivaroxaban to be cost-effective compared with enoxaparin + VKA approximately 76% of the time. LIMITATIONS: The model did not account for the benefits associated with an oral and minimally invasive administration of rivaroxaban. 'Real-world' applicability is limited because data from the EINSTEIN trials were used in the model. Also, resource utilization and costs were based on the US healthcare system. CONCLUSION: Rivaroxaban is a cost-effective option for anticoagulation treatment of acute VTE patients.