How Artificial Intelligence Will Transform Clinical Care, Research, and Trials for Inflammatory Bowel Disease.

Artificial intelligence (AI) refers to computer-based methodologies which use data to teach a computer to solve pre-defined tasks; these methods can be applied to identify patterns in large multi-modal data sources. AI applications in inflammatory bowel disease (IBD) includes predicting response to therapy, disease activity scoring of endoscopy, drug discovery, and identifying bowel damage in images. As a complex disease with entangled relationships between genomics, metabolomics, microbiome, and the environment, IBD stands to benefit greatly from methodologies that can handle this complexity. We describe current applications, critical challenges, and propose future directions of AI in IBD.

Validation of an Electronic Health Record-Based Machine Learning Model Compared to Clinical Risk Scores for Gastrointestinal Bleeding.

BACKGROUND & AIMS: Guidelines recommend use of risk stratification scores for patients presenting with gastrointestinal bleeding (GIB) to identify very-low-risk patients eligible for discharge from emergency departments. Machine learning models may outperform existing scores and can be integrated within the electronic health record (EHR) to provide real-time risk assessment without manual data entry. We present the first EHR-based machine learning model for GIB. METHODS: The training cohort comprised 2,546 patients and internal validation of 850 patients presenting with overt GIB (hematemesis, melena, hematochezia) to emergency departments of 2 hospitals from 2014-2019. External validation was performed on 926 patients presenting to a different hospital with the same EHR from 2014-2019. The primary outcome was a composite of red-blood-cell transfusion, hemostatic intervention (endoscopic, interventional radiologic, or surgical), and 30-day all-cause mortality. We used structured data fields in the EHR available within 4 hours of presentation and compared performance of machine learning models to current guideline-recommended risk scores, Glasgow-Blatchford Score (GBS) and Oakland Score. Primary analysis was area under the receiver-operating-characteristic curve (AUC). Secondary analysis was specificity at 99% sensitivity to assess proportion of patients correctly identified as very-low-risk. RESULTS: The machine learning model outperformed the GBS (AUC=0.92 vs. 0.89;p<0.001) and Oakland score (AUC=0.92 vs. 0.89;p<0.001). At the very-low-risk threshold of 99% sensitivity, the machine learning model identified more very-low-risk patients: 37.9% vs. 18.5% for GBS and 11.7% for Oakland score (p<0.001 for both comparisons). CONCLUSIONS: An EHR-based machine learning model performs better than currently recommended clinical risk scores and identifies more very-low-risk patients eligible for discharge from the emergency department.

Systematic review: The use of large language models as medical chatbots in digestive diseases.

BACKGROUND: Interest in large language models (LLMs), such as OpenAI's ChatGPT, across multiple specialties has grown as a source of patient-facing medical advice and provider-facing clinical decision support. The accuracy of LLM responses for gastroenterology and hepatology-related questions is unknown. AIMS: To evaluate the accuracy and potential safety implications for LLMs for the diagnosis, management and treatment of questions related to gastroenterology and hepatology. METHODS: We conducted a systematic literature search including Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus and the Web of Science Core Collection to identify relevant articles published from inception until January 28, 2024, using a combination of keywords and controlled vocabulary for LLMs and gastroenterology or hepatology. Accuracy was defined as the percentage of entirely correct answers. RESULTS: Among the 1671 reports screened, we identified 33 full-text articles on using LLMs in gastroenterology and hepatology and included 18 in the final analysis. The accuracy of question-responding varied across different model versions. For example, accuracy ranged from 6.4% to 45.5% with ChatGPT-3.5 and was between 40% and 91.4% with ChatGPT-4. In addition, the absence of standardised methodology and reporting metrics for studies involving LLMs places all the studies at a high risk of bias and does not allow for the generalisation of single-study results. CONCLUSIONS: Current general-purpose LLMs have unacceptably low accuracy on clinical gastroenterology and hepatology tasks, which may lead to adverse patient safety events through incorrect information or triage recommendations, which might overburden healthcare systems or delay necessary care.

Optimizing large language models in digestive disease: strategies and challenges to improve clinical outcomes.

Large Language Models (LLMs) are transformer-based neural networks with billions of parameters trained on very large text corpora from diverse sources. LLMs have the potential to improve healthcare due to their capability to parse complex concepts and generate context-based responses. The interest in LLMs has not spared digestive disease academics, who have mainly investigated foundational LLM accuracy, which ranges from 25% to 90% and is influenced by the lack of standardized rules to report methodologies and results for LLM-oriented research. In addition, a critical issue is the absence of a universally accepted definition of accuracy, varying from binary to scalar interpretations, often tied to grader expertise without reference to clinical guidelines. We address strategies and challenges to increase accuracy. In particular, LLMs can be infused with domain knowledge using Retrieval Augmented Generation (RAG) or Supervised Fine-Tuning (SFT) with reinforcement learning from human feedback (RLHF). RAG faces challenges with in-context window limits and accurate information retrieval from the provided context. SFT, a deeper adaptation method, is computationally demanding and requires specialized knowledge. LLMs may increase patient quality of care across the field of digestive diseases, where physicians are often engaged in screening, treatment and surveillance for a broad range of pathologies for which in-context learning or SFT with RLHF could improve clinical decision-making and patient outcomes. However, despite their potential, the safe deployment of LLMs in healthcare still needs to overcome hurdles in accuracy, suggesting a need for strategies that integrate human feedback with advanced model training.

Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation-based framework.

Large language models (LLMs) can potentially transform healthcare, particularly in providing the right information to the right provider at the right time in the hospital workflow. This study investigates the integration of LLMs into healthcare, specifically focusing on improving clinical decision support systems (CDSSs) through accurate interpretation of medical guidelines for chronic Hepatitis C Virus infection management. Utilizing OpenAI's GPT-4 Turbo model, we developed a customized LLM framework that incorporates retrieval augmented generation (RAG) and prompt engineering. Our framework involved guideline conversion into the best-structured format that can be efficiently processed by LLMs to provide the most accurate output. An ablation study was conducted to evaluate the impact of different formatting and learning strategies on the LLM's answer generation accuracy. The baseline GPT-4 Turbo model's performance was compared against five experimental setups with increasing levels of complexity: inclusion of in-context guidelines, guideline reformatting, and implementation of few-shot learning. Our primary outcome was the qualitative assessment of accuracy based on expert review, while secondary outcomes included the quantitative measurement of similarity of LLM-generated responses to expert-provided answers using text-similarity scores. The results showed a significant improvement in accuracy from 43 to 99% (p < 0.001), when guidelines were provided as context in a coherent corpus of text and non-text sources were converted into text. In addition, few-shot learning did not seem to improve overall accuracy. The study highlights that structured guideline reformatting and advanced prompt engineering (data quality vs. data quantity) can enhance the efficacy of LLM integrations to CDSSs for guideline delivery.

Predicting response to non-selective beta-blockers with liver-spleen stiffness and heart rate in patients with liver cirrhosis and high-risk varices.

INTRODUCTION: Non-selective beta-blockers (NSBB) are used for primary prophylaxis in patients with liver cirrhosis and high-risk varices (HRVs). Assessing therapeutic response is challenging due to the invasive nature of hepatic venous pressure gradient (HVPG) measurement. This study aims to define a noninvasive machine-learning based approach to determine response to NSBB in patients with liver cirrhosis and HRVs. METHODS: We conducted a prospective study on a cohort of cirrhotic patients with documented HRVs receiving NSBB treatment. Patients were followed-up with clinical and elastography appointments at 3, 6, and 12 months after NSBB treatment initiation. NSBB response was defined as stationary or downstaging variceal grading at the 12-month esophagogastroduodenoscopy (EGD). In contrast, non-response was defined as upstaging variceal grading at the 12-month EGD or at least one variceal hemorrhage episode during the 12-month follow-up. We chose cut-off values for univariate and multivariate model with 100% specificity. RESULTS: According to least absolute shrinkage and selection operator (LASSO) regression, spleen stiffness (SS) and liver stiffness (LS) percentual decrease, along with changes in heart rate (HR) at 3 months were the most significant predictors of NSBB response. A decrease > 11.5% in SS, > 16.8% in LS, and > 25.3% in HR was associated with better prediction of clinical response to NSBB. SS percentual decrease showed the highest accuracy (86.4%) with high sensitivity (78.8%) when compared to LS and HR. The multivariate model incorporating SS, LS, and HR showed the highest discrimination and calibration metrics (AUROC = 0.96), with the optimal cut-off of 0.90 (sensitivity 94.2%, specificity 100%, PPV 95.7%, NPV 100%, accuracy 97.5%).

Review article: Upper gastrointestinal bleeding - review of current evidence and implications for management.

BACKGROUND: Acute upper gastrointestinal bleeding (UGIB) is a common emergency requiring hospital-based care. Advances in care across pre-endoscopic, endoscopic and post-endoscopic phases have led to improvements in clinical outcomes. AIMS: To provide a detailed, evidence-based update on major aspects of care across pre-endoscopic, endoscopic and post-endoscopic phases. METHODS: We performed a structured bibliographic database search for each topic. If a recent high-quality meta-analysis was not available, we performed a meta-analysis with random effects methods and odds ratios with 95% confidence intervals. RESULTS: Pre-endoscopic management of UGIB includes risk stratification, a restrictive red blood cell transfusion policy unless the patient has cardiovascular disease, and pharmacologic therapy with erythromycin and a proton pump inhibitor. Patients with cirrhosis should be treated with prophylactic antibiotics and vasoactive medications. Tranexamic acid should not be used. Endoscopic management of UGIB depends on the aetiology. For peptic ulcer disease (PUD) with high-risk stigmata, endoscopic therapy, including over-the-scope clips (OTSCs) and TC-325 powder spray, should be performed. For variceal bleeding, treatment should be customised by severity and anatomic location. Post-endoscopic management includes early enteral feeding for all UGIB patients. For high-risk PUD, PPI should be continued for 72 h, and rebleeding should initially be evaluated with a repeat endoscopy. For variceal bleeding, high-risk patients or those with further bleeding, a transjugular intrahepatic portosystemic shunt can be considered. CONCLUSIONS: Management of acute UGIB should include treatment plans for pre-endoscopic, endoscopic and post-endoscopic phases of care, and customise treatment decisions based on aetiology and severity of bleeding.

Achieving Value by Risk Stratification With Machine Learning Model or Clinical Risk Score in Acute Upper Gastrointestinal Bleeding: A Cost Minimization Analysis.

INTRODUCTION: We estimate the economic impact of applying risk assessment tools to identify very low-risk patients with upper gastrointestinal bleeding who can be safely discharged from the emergency department using a cost minimization analysis. METHODS: We compare triage strategies (Glasgow-Blatchford score = 0/0-1 or validated machine learning model) with usual care using a Markov chain model from a US health care payer perspective. RESULTS: Over 5 years, the Glasgow-Blatchford score triage strategy produced national cumulative savings over usual care of more than $2.7 billion and the machine learning strategy of more than $3.4 billion. DISCUSSION: Implementing risk assessment models for upper gastrointestinal bleeding reduces costs, thereby increasing value.

Harnessing the power of synthetic data in healthcare: innovation, application, and privacy.

Data-driven decision-making in modern healthcare underpins innovation and predictive analytics in public health and clinical research. Synthetic data has shown promise in finance and economics to improve risk assessment, portfolio optimization, and algorithmic trading. However, higher stakes, potential liabilities, and healthcare practitioner distrust make clinical use of synthetic data difficult. This paper explores the potential benefits and limitations of synthetic data in the healthcare analytics context. We begin with real-world healthcare applications of synthetic data that informs government policy, enhance data privacy, and augment datasets for predictive analytics. We then preview future applications of synthetic data in the emergent field of digital twin technology. We explore the issues of data quality and data bias in synthetic data, which can limit applicability across different applications in the clinical context, and privacy concerns stemming from data misuse and risk of re-identification. Finally, we evaluate the role of regulatory agencies in promoting transparency and accountability and propose strategies for risk mitigation such as Differential Privacy (DP) and a dataset chain of custody to maintain data integrity, traceability, and accountability. Synthetic data can improve healthcare, but measures to protect patient well-being and maintain ethical standards are key to promote responsible use.

Trends in Upper Gastrointestinal Bleeding in Patients on Primary Prevention Aspirin: A Nationwide Emergency Department Sample Analysis, 2016-2020.

BACKGROUND: Recent guidelines do not recommend routine use of aspirin for primary cardiovascular prevention (ppASA) and suggest avoidance of ppASA in older individuals due to bleeding risk. However, ppASA is frequently taken without an appropriate indication. Estimates of the incidence of upper gastrointestinal bleeding due to ppASA in the United States are lacking. In this study, we provide national estimates of upper gastrointestinal bleeding incidence, characteristics, and costs in ppASA users from 2016-2020. METHODS: Primary cardiovascular prevention users (patients on long-term aspirin therapy without cardiovascular disease) presenting with upper gastrointestinal bleeding were identified in the Nationwide Emergency Department Sample using International Statistical Classification of Diseases and Related Health Problems, 10th revision codes. Trends in upper gastrointestinal bleeding incidence, etiology, severity, associated Medicare reimbursements, and the impact of ppASA on bleeding outcomes were assessed with regression models. RESULTS: From 2016-2020, adjusted incidence of upper gastrointestinal bleeding increased 29.2% among ppASA users, with larger increases for older patients (increase of 41.6% for age 65-74 years and 36.0% for age ≥75 years). The most common etiology among ppASA users was ulcer disease but increases in bleeding incidence due to angiodysplasias were observed. The proportion of hospitalizations with major complications or comorbidities increased 41.5%, and Medicare reimbursements increased 67.6%. Among patients without cardiovascular disease, ppASA was associated with increased odds of hospital admission, red blood cell transfusion, and endoscopic intervention as compared to no ppASA use. CONCLUSIONS: Considering recent guideline recommendations, the rising incidence, severity, and costs associated with upper gastrointestinal bleeding among patients on ppASA highlights the importance of careful assessment for appropriate ppASA use.

Trends in characteristics, management, and outcomes of patients presenting with gastrointestinal bleeding to emergency departments in the United States from 2006 to 2019.

BACKGROUND: Recent epidemiologic studies of trends in gastrointestinal bleeding (GIB) provided results through 2014 or earlier and assessed only hospitalised patients, excluding patients presenting to emergency departments (EDs) who are not hospitalised. AIMS: To provide the first U.S. nationwide epidemiological evaluation of all patients presenting to EDs with GIB METHODS: We used the Nationwide Emergency Department Sample for 2006-2019 to calculate yearly projected incidence of patients presenting to EDs with primary diagnoses of GIB, categorised by location and aetiology. Outcomes were assessed with multivariable analyses. RESULTS: The age/sex-adjusted incidence for GIB increased from 378.4 to 397.5/100,000 population from 2006 to 2019. Upper gastrointestinal bleeding (UGIB) incidence decreased from 2006 to 2014 (112.3-94.4/100,000) before increasing to 116.2/100,000 by 2019. Lower gastrointestinal bleeding (LGIB) incidence increased from 2006 to 2015 (146.0 to 161.0/100,000) before declining to 150.2/100,000 by 2019. The proportion of cases with one or more comorbidities increased from 27.4% to 35.9% from 2006 to 2019. Multivariable analyses comparing 2019 to 2006 showed increases in ED discharges (odds ratio [OR] = 1.45; 95% confidence interval [CI] = 1.43-1.48) and decreases in red blood cell (RBC) transfusions (OR = 0.62; 0.61-0.63), endoscopies (OR = 0.60; 0.59-0.61), death (OR = 0.51; 0.48-0.54) and length of stay (relative ratio [RR] = 0.81; 0.80-0.82). Inpatient cost decreased from 2012 to 2019 (RR = 0.92; 0.91-0.93). CONCLUSIONS: The incidence of GIB in the U.S. is increasing. UGIB incidence has been increasing since 2014 while LGIB incidence has been decreasing since 2015. Despite a more comorbid population in 2019, case fatality rate, length of stay and costs have decreased. More patients are discharged from the ED and the rate of RBC transfusions has decreased, possibly reflecting changing clinical practice in response to updated guidelines.

Randomized Clinical Trials of Machine Learning Interventions in Health Care: A Systematic Review.

Importance: Despite the potential of machine learning to improve multiple aspects of patient care, barriers to clinical adoption remain. Randomized clinical trials (RCTs) are often a prerequisite to large-scale clinical adoption of an intervention, and important questions remain regarding how machine learning interventions are being incorporated into clinical trials in health care. Objective: To systematically examine the design, reporting standards, risk of bias, and inclusivity of RCTs for medical machine learning interventions. Evidence Review: In this systematic review, the Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus, and Web of Science Core Collection online databases were searched and citation chasing was done to find relevant articles published from the inception of each database to October 15, 2021. Search terms for machine learning, clinical decision-making, and RCTs were used. Exclusion criteria included implementation of a non-RCT design, absence of original data, and evaluation of nonclinical interventions. Data were extracted from published articles. Trial characteristics, including primary intervention, demographics, adherence to the CONSORT-AI reporting guideline, and Cochrane risk of bias were analyzed. Findings: Literature search yielded 19 737 articles, of which 41 RCTs involved a median of 294 participants (range, 17-2488 participants). A total of 16 RCTS (39%) were published in 2021, 21 (51%) were conducted at single sites, and 15 (37%) involved endoscopy. No trials adhered to all CONSORT-AI standards. Common reasons for nonadherence were not assessing poor-quality or unavailable input data (38 trials [93%]), not analyzing performance errors (38 [93%]), and not including a statement regarding code or algorithm availability (37 [90%]). Overall risk of bias was high in 7 trials (17%). Of 11 trials (27%) that reported race and ethnicity data, the median proportion of participants from underrepresented minority groups was 21% (range, 0%-51%). Conclusions and Relevance: This systematic review found that despite the large number of medical machine learning-based algorithms in development, few RCTs for these technologies have been conducted. Among published RCTs, there was high variability in adherence to reporting standards and risk of bias and a lack of participants from underrepresented minority groups. These findings merit attention and should be considered in future RCT design and reporting.

Randomized Controlled Trials of Artificial Intelligence in Clinical Practice: Systematic Review.

BACKGROUND: The number of artificial intelligence (AI) studies in medicine has exponentially increased recently. However, there is no clear quantification of the clinical benefits of implementing AI-assisted tools in patient care. OBJECTIVE: This study aims to systematically review all published randomized controlled trials (RCTs) of AI-assisted tools to characterize their performance in clinical practice. METHODS: CINAHL, Cochrane Central, Embase, MEDLINE, and PubMed were searched to identify relevant RCTs published up to July 2021 and comparing the performance of AI-assisted tools with conventional clinical management without AI assistance. We evaluated the primary end points of each study to determine their clinical relevance. This systematic review was conducted following the updated PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 guidelines. RESULTS: Among the 11,839 articles retrieved, only 39 (0.33%) RCTs were included. These RCTs were conducted in an approximately equal distribution from North America, Europe, and Asia. AI-assisted tools were implemented in 13 different clinical specialties. Most RCTs were published in the field of gastroenterology, with 15 studies on AI-assisted endoscopy. Most RCTs studied biosignal-based AI-assisted tools, and a minority of RCTs studied AI-assisted tools drawn from clinical data. In 77% (30/39) of the RCTs, AI-assisted interventions outperformed usual clinical care, and clinically relevant outcomes improved with AI-assisted intervention in 70% (21/30) of the studies. Small sample size and single-center design limited the generalizability of these studies. CONCLUSIONS: There is growing evidence supporting the implementation of AI-assisted tools in daily clinical practice; however, the number of available RCTs is limited and heterogeneous. More RCTs of AI-assisted tools integrated into clinical practice are needed to advance the role of AI in medicine. TRIAL REGISTRATION: PROSPERO CRD42021286539; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=286539.