Validation of an Electronic Health Record-Based Machine Learning Model Compared to Clinical Risk Scores for Gastrointestinal Bleeding.

BACKGROUND & AIMS: Guidelines recommend use of risk stratification scores for patients presenting with gastrointestinal bleeding (GIB) to identify very-low-risk patients eligible for discharge from emergency departments. Machine learning models may outperform existing scores and can be integrated within the electronic health record (EHR) to provide real-time risk assessment without manual data entry. We present the first EHR-based machine learning model for GIB. METHODS: The training cohort comprised 2,546 patients and internal validation of 850 patients presenting with overt GIB (hematemesis, melena, hematochezia) to emergency departments of 2 hospitals from 2014-2019. External validation was performed on 926 patients presenting to a different hospital with the same EHR from 2014-2019. The primary outcome was a composite of red-blood-cell transfusion, hemostatic intervention (endoscopic, interventional radiologic, or surgical), and 30-day all-cause mortality. We used structured data fields in the EHR available within 4 hours of presentation and compared performance of machine learning models to current guideline-recommended risk scores, Glasgow-Blatchford Score (GBS) and Oakland Score. Primary analysis was area under the receiver-operating-characteristic curve (AUC). Secondary analysis was specificity at 99% sensitivity to assess proportion of patients correctly identified as very-low-risk. RESULTS: The machine learning model outperformed the GBS (AUC=0.92 vs. 0.89;p<0.001) and Oakland score (AUC=0.92 vs. 0.89;p<0.001). At the very-low-risk threshold of 99% sensitivity, the machine learning model identified more very-low-risk patients: 37.9% vs. 18.5% for GBS and 11.7% for Oakland score (p<0.001 for both comparisons). CONCLUSIONS: An EHR-based machine learning model performs better than currently recommended clinical risk scores and identifies more very-low-risk patients eligible for discharge from the emergency department.

Optimizing large language models in digestive disease: strategies and challenges to improve clinical outcomes.

Large Language Models (LLMs) are transformer-based neural networks with billions of parameters trained on very large text corpora from diverse sources. LLMs have the potential to improve healthcare due to their capability to parse complex concepts and generate context-based responses. The interest in LLMs has not spared digestive disease academics, who have mainly investigated foundational LLM accuracy, which ranges from 25% to 90% and is influenced by the lack of standardized rules to report methodologies and results for LLM-oriented research. In addition, a critical issue is the absence of a universally accepted definition of accuracy, varying from binary to scalar interpretations, often tied to grader expertise without reference to clinical guidelines. We address strategies and challenges to increase accuracy. In particular, LLMs can be infused with domain knowledge using Retrieval Augmented Generation (RAG) or Supervised Fine-Tuning (SFT) with reinforcement learning from human feedback (RLHF). RAG faces challenges with in-context window limits and accurate information retrieval from the provided context. SFT, a deeper adaptation method, is computationally demanding and requires specialized knowledge. LLMs may increase patient quality of care across the field of digestive diseases, where physicians are often engaged in screening, treatment and surveillance for a broad range of pathologies for which in-context learning or SFT with RLHF could improve clinical decision-making and patient outcomes. However, despite their potential, the safe deployment of LLMs in healthcare still needs to overcome hurdles in accuracy, suggesting a need for strategies that integrate human feedback with advanced model training.

Systematic review: The use of large language models as medical chatbots in digestive diseases.

BACKGROUND: Interest in large language models (LLMs), such as OpenAI's ChatGPT, across multiple specialties has grown as a source of patient-facing medical advice and provider-facing clinical decision support. The accuracy of LLM responses for gastroenterology and hepatology-related questions is unknown. AIMS: To evaluate the accuracy and potential safety implications for LLMs for the diagnosis, management and treatment of questions related to gastroenterology and hepatology. METHODS: We conducted a systematic literature search including Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus and the Web of Science Core Collection to identify relevant articles published from inception until January 28, 2024, using a combination of keywords and controlled vocabulary for LLMs and gastroenterology or hepatology. Accuracy was defined as the percentage of entirely correct answers. RESULTS: Among the 1671 reports screened, we identified 33 full-text articles on using LLMs in gastroenterology and hepatology and included 18 in the final analysis. The accuracy of question-responding varied across different model versions. For example, accuracy ranged from 6.4% to 45.5% with ChatGPT-3.5 and was between 40% and 91.4% with ChatGPT-4. In addition, the absence of standardised methodology and reporting metrics for studies involving LLMs places all the studies at a high risk of bias and does not allow for the generalisation of single-study results. CONCLUSIONS: Current general-purpose LLMs have unacceptably low accuracy on clinical gastroenterology and hepatology tasks, which may lead to adverse patient safety events through incorrect information or triage recommendations, which might overburden healthcare systems or delay necessary care.

Predicting response to non-selective beta-blockers with liver-spleen stiffness and heart rate in patients with liver cirrhosis and high-risk varices.

INTRODUCTION: Non-selective beta-blockers (NSBB) are used for primary prophylaxis in patients with liver cirrhosis and high-risk varices (HRVs). Assessing therapeutic response is challenging due to the invasive nature of hepatic venous pressure gradient (HVPG) measurement. This study aims to define a noninvasive machine-learning based approach to determine response to NSBB in patients with liver cirrhosis and HRVs. METHODS: We conducted a prospective study on a cohort of cirrhotic patients with documented HRVs receiving NSBB treatment. Patients were followed-up with clinical and elastography appointments at 3, 6, and 12 months after NSBB treatment initiation. NSBB response was defined as stationary or downstaging variceal grading at the 12-month esophagogastroduodenoscopy (EGD). In contrast, non-response was defined as upstaging variceal grading at the 12-month EGD or at least one variceal hemorrhage episode during the 12-month follow-up. We chose cut-off values for univariate and multivariate model with 100% specificity. RESULTS: According to least absolute shrinkage and selection operator (LASSO) regression, spleen stiffness (SS) and liver stiffness (LS) percentual decrease, along with changes in heart rate (HR) at 3 months were the most significant predictors of NSBB response. A decrease > 11.5% in SS, > 16.8% in LS, and > 25.3% in HR was associated with better prediction of clinical response to NSBB. SS percentual decrease showed the highest accuracy (86.4%) with high sensitivity (78.8%) when compared to LS and HR. The multivariate model incorporating SS, LS, and HR showed the highest discrimination and calibration metrics (AUROC = 0.96), with the optimal cut-off of 0.90 (sensitivity 94.2%, specificity 100%, PPV 95.7%, NPV 100%, accuracy 97.5%).

Single-cell analysis reveals inflammatory interactions driving macular degeneration.

Due to commonalities in pathophysiology, age-related macular degeneration (AMD) represents a uniquely accessible model to investigate therapies for neurodegenerative diseases, leading us to examine whether pathways of disease progression are shared across neurodegenerative conditions. Here we use single-nucleus RNA sequencing to profile lesions from 11 postmortem human retinas with age-related macular degeneration and 6 control retinas with no history of retinal disease. We create a machine-learning pipeline based on recent advances in data geometry and topology and identify activated glial populations enriched in the early phase of disease. Examining single-cell data from Alzheimer's disease and progressive multiple sclerosis with our pipeline, we find a similar glial activation profile enriched in the early phase of these neurodegenerative diseases. In late-stage age-related macular degeneration, we identify a microglia-to-astrocyte signaling axis mediated by interleukin-1ß which drives angiogenesis characteristic of disease pathogenesis. We validated this mechanism using in vitro and in vivo assays in mouse, identifying a possible new therapeutic target for AMD and possibly other neurodegenerative conditions. Thus, due to shared glial states, the retina provides a potential system for investigating therapeutic approaches in neurodegenerative diseases.

Geometric learning of functional brain network on the correlation manifold.

The correlation matrix is a typical representation of node interactions in functional brain network analysis. The analysis of the correlation matrix to characterize brain networks observed in several neuroimaging modalities has been conducted predominantly in the Euclidean space by assuming that pairwise interactions are mutually independent. One way to take account of all interactions in the network as a whole is to analyze the correlation matrix under some geometric structure. Recent studies have focused on the space of correlation matrices as a strict subset of symmetric positive definite (SPD) matrices, which form a unique mathematical structure known as the Riemannian manifold. However, mathematical operations of the correlation matrix under the SPD geometry may not necessarily be coherent (i.e., the structure of the correlation matrix may not be preserved), necessitating a post-hoc normalization. The contribution of the current paper is twofold: (1) to devise a set of inferential methods on the correlation manifold and (2) to demonstrate its applicability in functional network analysis. We present several algorithms on the correlation manifold, including measures of central tendency, cluster analysis, hypothesis testing, and low-dimensional embedding. Simulation and real data analysis support the application of the proposed framework for brain network analysis.

Re-visiting Riemannian geometry of symmetric positive definite matrices for the analysis of functional connectivity.

Common representations of functional networks of resting state fMRI time series, including covariance, precision, and cross-correlation matrices, belong to the family of symmetric positive definite (SPD) matrices forming a special mathematical structure called Riemannian manifold. Due to its geometric properties, the analysis and operation of functional connectivity matrices may well be performed on the Riemannian manifold of the SPD space. Analysis of functional networks on the SPD space takes account of all the pairwise interactions (edges) as a whole, which differs from the conventional rationale of considering edges as independent from each other. Despite its geometric characteristics, only a few studies have been conducted for functional network analysis on the SPD manifold and inference methods specialized for connectivity analysis on the SPD manifold are rarely found. The current study aims to show the significance of connectivity analysis on the SPD space and introduce inference algorithms on the SPD manifold, such as regression analysis of functional networks in association with behaviors, principal geodesic analysis, clustering, state transition analysis of dynamic functional networks and statistical tests for network equality on the SPD manifold. We applied the proposed methods to both simulated data and experimental resting state fMRI data from the human connectome project and argue the importance of analyzing functional networks under the SPD geometry. All the algorithms for numerical operations and inferences on the SPD manifold are implemented as a MATLAB library, called SPDtoolbox, for public use to expediate functional network analysis on the right geometry.