Systematic review: The use of large language models as medical chatbots in digestive diseases.

BACKGROUND: Interest in large language models (LLMs), such as OpenAI's ChatGPT, across multiple specialties has grown as a source of patient-facing medical advice and provider-facing clinical decision support. The accuracy of LLM responses for gastroenterology and hepatology-related questions is unknown. AIMS: To evaluate the accuracy and potential safety implications for LLMs for the diagnosis, management and treatment of questions related to gastroenterology and hepatology. METHODS: We conducted a systematic literature search including Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus and the Web of Science Core Collection to identify relevant articles published from inception until January 28, 2024, using a combination of keywords and controlled vocabulary for LLMs and gastroenterology or hepatology. Accuracy was defined as the percentage of entirely correct answers. RESULTS: Among the 1671 reports screened, we identified 33 full-text articles on using LLMs in gastroenterology and hepatology and included 18 in the final analysis. The accuracy of question-responding varied across different model versions. For example, accuracy ranged from 6.4% to 45.5% with ChatGPT-3.5 and was between 40% and 91.4% with ChatGPT-4. In addition, the absence of standardised methodology and reporting metrics for studies involving LLMs places all the studies at a high risk of bias and does not allow for the generalisation of single-study results. CONCLUSIONS: Current general-purpose LLMs have unacceptably low accuracy on clinical gastroenterology and hepatology tasks, which may lead to adverse patient safety events through incorrect information or triage recommendations, which might overburden healthcare systems or delay necessary care.

Predicting response to non-selective beta-blockers with liver-spleen stiffness and heart rate in patients with liver cirrhosis and high-risk varices.

INTRODUCTION: Non-selective beta-blockers (NSBB) are used for primary prophylaxis in patients with liver cirrhosis and high-risk varices (HRVs). Assessing therapeutic response is challenging due to the invasive nature of hepatic venous pressure gradient (HVPG) measurement. This study aims to define a noninvasive machine-learning based approach to determine response to NSBB in patients with liver cirrhosis and HRVs. METHODS: We conducted a prospective study on a cohort of cirrhotic patients with documented HRVs receiving NSBB treatment. Patients were followed-up with clinical and elastography appointments at 3, 6, and 12 months after NSBB treatment initiation. NSBB response was defined as stationary or downstaging variceal grading at the 12-month esophagogastroduodenoscopy (EGD). In contrast, non-response was defined as upstaging variceal grading at the 12-month EGD or at least one variceal hemorrhage episode during the 12-month follow-up. We chose cut-off values for univariate and multivariate model with 100% specificity. RESULTS: According to least absolute shrinkage and selection operator (LASSO) regression, spleen stiffness (SS) and liver stiffness (LS) percentual decrease, along with changes in heart rate (HR) at 3 months were the most significant predictors of NSBB response. A decrease > 11.5% in SS, > 16.8% in LS, and > 25.3% in HR was associated with better prediction of clinical response to NSBB. SS percentual decrease showed the highest accuracy (86.4%) with high sensitivity (78.8%) when compared to LS and HR. The multivariate model incorporating SS, LS, and HR showed the highest discrimination and calibration metrics (AUROC = 0.96), with the optimal cut-off of 0.90 (sensitivity 94.2%, specificity 100%, PPV 95.7%, NPV 100%, accuracy 97.5%).