Evaluation and mitigation of the limitations of large language models in clinical decision-making.

Clinical decision-making is one of the most impactful parts of a physician's responsibilities and stands to benefit greatly from artificial intelligence solutions and large language models (LLMs) in particular. However, while LLMs have achieved excellent performance on medical licensing exams, these tests fail to assess many skills necessary for deployment in a realistic clinical decision-making environment, including gathering information, adhering to guidelines, and integrating into clinical workflows. Here we have created a curated dataset based on the Medical Information Mart for Intensive Care database spanning 2,400 real patient cases and four common abdominal pathologies as well as a framework to simulate a realistic clinical setting. We show that current state-of-the-art LLMs do not accurately diagnose patients across all pathologies (performing significantly worse than physicians), follow neither diagnostic nor treatment guidelines, and cannot interpret laboratory results, thus posing a serious risk to the health of patients. Furthermore, we move beyond diagnostic accuracy and demonstrate that they cannot be easily integrated into existing workflows because they often fail to follow instructions and are sensitive to both the quantity and order of information. Overall, our analysis reveals that LLMs are currently not ready for autonomous clinical decision-making while providing a dataset and framework to guide future studies.

Severity of gallstone, sludge or microlithiasis induced pancreatitis - all of the same?

BACKGROUND/AIM: Severity of microlithiasis and sludge-induced pancreatitis in comparison to gallstone-induced pancreatitis has never been studied for a lack of definition. In order to understand whether bile duct obstruction or other mechanisms contribute to biliary pancreatitis severity we performed a monocentric, retrospective cohort study. METHODS: In this retrospective cohort study 263 patients with acute biliary pancreatitis treated at a tertiary care center from 2005 to 2021 were stratified according to the recent consensus definition for microlithiasis and sludge. The gallstone-pancreatitis cohort was compared to microlithiasis, sludge and suspected stone passage pancreatitis cohorts in terms of pancreatitis outcome, liver function and EUS/ERCP results using one-way ANOVA and Chi2 test. Multinomial logistic regression analysis was performed to correct for bias. RESULTS: Microlithiasis and sludge-induced pancreatitis classified according to the revised Atlanta classification, did not present with a milder course than gallstone-induced pancreatitis (p = 0.62). Microlithiasis and sludge showed an increase in bilirubin on the day of admission to hospital, which was not significantly different from gallstone-induced pancreatitis (p = 0.36). The likelihood of detecting biliary disease on EUS resulting in bile duct clearance was highest on the day of admission and day 1, respectively. CONCLUSION: Microlithiasis and sludge induce gallstone-equivalent impaired liver function tests and induce pancreatitis with similar severity compared with gallstone-induced acute biliary pancreatitis.

Concomitant Irradiation to Checkpoint Inhibitor Therapy of Hepatocellular Carcinoma Patients: A Systematic Retrospective, Single-Center Analysis.

INTRODUCTION: Immunotherapy has been established as the standard treatment option for patients with advanced hepatocellular carcinoma (aHCC). Despite the increased efficacy, disease progression occurs in a relevant proportion of patients even after an objective response. Combination concepts with locoregional therapy are currently under investigation for hepatic disease but are also in discussion for the control of distant metastasis. Radiotherapy is a highly effective treatment modality for local tumor control. It is also thought to increase the efficacy of checkpoint inhibition and sensitize distant lesions to the effects of immunotherapy, but may potentially increase adverse effects. In our center, few patients with aHCC treated with immune checkpoint inhibitors (ICIs) received concomitant radiotherapy for symptom or disease control. The aim of this study was to retrospectively analyze adverse effects and efficacy of concomitant radiotherapy in patients with aHCC treated with checkpoint inhibition. METHODS: To this aim, patients who received a combination of ICI and radiotherapy in our institution were retrospectively considered for analysis. The predefined inclusion criterion was radiotherapy after initiated checkpoint inhibition and continuation of ICI therapy for at least 8 weeks. Adverse effects and efficacy measurements were performed according to local standards. RESULTS: The database search of 2016-2021 revealed six consecutive patients fulfilling the predefined criteria for concomitant ICI and radiotherapy. Three patients received high-dose-rate brachytherapy (15 Gy) to treat progredient hepatic lesions. Two patients received stereotactic body radiotherapy (SBRT) (25-30 Gy) for symptom control, and 1 patient received brachytherapy and SBRT to treat metastases. No severe adverse events were reported in the period (<6 months) after concomitant radiotherapy. In 5 out of 6 cases, long-term tumor control could be achieved by this therapeutic combination. CONCLUSION: A good efficacy of concomitant radiotherapy and checkpoint inhibition has been achieved with no safety concerns. Further investigations should evaluate the safety, appropriate clinical context, and efficacy of this promising approach.

How to exclude pulmonary embolism in patients hospitalized with COVID-19: a comparison of predictive scores.

BACKGROUND: Pulmonary embolism (PE) is an important complication of Coronavirus disease 2019 (COVID-19). COVID-19 is associated with respiratory impairment and a pro-coagulative state, rendering PE more likely and difficult to recognize. Several decision algorithms relying on clinical features and D-dimer have been established. High prevalence of PE and elevated Ddimer in patients with COVID-19 might impair the performance of common decision algorithms. Here, we aimed to validate and compare five common decision algorithms implementing age adjusted Ddimer, the GENEVA, and Wells scores as well as the PEGeD- and YEARS-algorithms in patients hospitalized with COVID-19. METHODS: In this single center study, we included patients who were admitted to our tertiary care hospital in the COVID-19 Registry of the LMU Munich. We retrospectively selected patients who received a computed tomography pulmonary angiogram (CTPA) or pulmonary ventilation/perfusion scintigraphy (V/Q) for suspected PE. The performances of five commonly used diagnostic algorithms (age-adjusted D-dimer, GENEVA score, PEGeD-algorithm, Wells score, and YEARS-algorithm) were compared. RESULTS: We identified 413 patients with suspected PE who received a CTPA or V/Q confirming 62 PEs (15%). Among them, 358 patients with 48 PEs (13%) could be evaluated for performance of all algorithms. Patients with PE were older and their overall outcome was worse compared to patients without PE. Of the above five diagnostic algorithms, the PEGeD- and YEARS-algorithms performed best, reducing diagnostic imaging by 14% and 15% respectively with a sensitivity of 95.7% and 95.6%. The GENEVA score was able to reduce CTPA or V/Q by 32.2% but suffered from a low sensitivity (78.6%). Age-adjusted D-dimer and Wells score could not significantly reduce diagnostic imaging. CONCLUSION: The PEGeD- and YEARS-algorithms outperformed other tested decision algorithms and worked well in patients admitted with COVID-19. These findings need independent validation in a prospective study.

Validation of the SACOV-19 score for identifying patients at risk of complicated or more severe COVID-19: a prospective study.

PURPOSE: Identification of patients at risk of complicated or more severe COVID-19 is of pivotal importance, since these patients might require monitoring, antiviral treatment, and hospitalization. In this study, we prospectively evaluated the SACOV-19 score for its ability to predict complicated or more severe COVID-19. METHODS: In this prospective multicenter study, we included 124 adult patients with acute COVID-19 in three German hospitals, who were diagnosed in an early, uncomplicated stage of COVID-19 within 72 h of inclusion. We determined the SACOV-19 score at baseline and performed a follow-up at 30 days. RESULTS: The SACOV-19 score's AUC was 0.816. At a cutoff of > 3, it predicted deterioration to complicated or more severe COVID-19 with a sensitivity of 94% and a specificity of 55%. It performed significantly better in predicting complicated COVID-19 than the random tree-based SACOV-19 predictive model, the CURB-65, 4C mortality, or qCSI scores. CONCLUSION: The SACOV-19 score is a feasible tool to aid decision making in acute COVID-19.

Independent Validation and Assay Standardization of Improved Metabolic Biomarker Signature to Differentiate Pancreatic Ductal Adenocarcinoma From Chronic Pancreatitis.

BACKGROUND & AIMS: Pancreatic ductal adenocarcinoma cancer (PDAC) is a highly lethal malignancy requiring efficient detection when the primary tumor is still resectable. We previously developed the MxPancreasScore comprising 9 analytes and serum carbohydrate antigen 19-9 (CA19-9), achieving an accuracy of 90.6%. The necessity for 5 different analytical platforms and multiple analytical runs, however, hindered clinical applicability. We therefore aimed to develop a simpler single-analytical run, single-platform diagnostic signature. METHODS: We evaluated 941 patients (PDAC, 356; chronic pancreatitis [CP], 304; nonpancreatic disease, 281) in 3 multicenter independent tests, and identification (ID) and validation cohort 1 (VD1) and 2 (VD2) were evaluated. Targeted quantitative plasma metabolite analysis was performed on a liquid chromatography-tandem mass spectrometry platform. A machine learning-aided algorithm identified an improved (i-Metabolic) and minimalistic metabolic (m-Metabolic) signatures, and compared them for performance. RESULTS: The i-Metabolic Signature, (12 analytes plus CA19-9) distinguished PDAC from CP with area under the curve (95% confidence interval) of 97.2% (97.1%-97.3%), 93.5% (93.4%-93.7%), and 92.2% (92.1%-92.3%) in the ID, VD1, and VD2 cohorts, respectively. In the VD2 cohort, the m-Metabolic signature (4 analytes plus CA19-9) discriminated PDAC from CP with a sensitivity of 77.3% and specificity of 89.6%, with an overall accuracy of 82.4%. For the subset of 45 patients with PDAC with resectable stages IA-IIB tumors, the sensitivity, specificity, and accuracy were 73.2%, 89.6%, and 82.7%, respectively; for those with detectable CA19-9 >2 U/mL, 81.6%, 88.7%, and 84.5%, respectively; and for those with CA19-9 <37 U/mL, 39.7%, 94.1%, and 76.3%, respectively. CONCLUSIONS: The single-platform, single-run, m-Metabolic signature of just 4 metabolites used in combination with serum CA19-9 levels is an innovative accurate diagnostic tool for PDAC at the time of clinical presentation, warranting further large-scale evaluation.

The Immunometabolomic Interface Receptor Hydroxycarboxylic Acid Receptor 2 Mediates the Therapeutic Effects of Dimethyl Fumarate in Autoantibody-Induced Skin Inflammation.

The drug dimethyl fumarate (DMF) is in clinical use for the treatment of psoriasis and multiple sclerosis. In addition, it has recently been demonstrated to ameliorate skin pathology in mouse models of pemphigoid diseases, a group of autoimmune blistering diseases of the skin and mucous membranes. However, the mode of action of DMF in inflammatory skin diseases has remained elusive. Therefore, we have investigated here the mechanisms by which DMF improves skin pathology, using the antibody transfer model of bullous pemphigoid-like epidermolysis bullosa acquisita (EBA). Experimental EBA was induced by transfer of antibodies against collagen VII that triggered the infiltration of immune cells into the skin and led to inflammatory skin lesions. DMF treatment reduced the infiltration of neutrophils and monocytes into the skin explaining the improved disease outcome in DMF-treated animals. Upon ingestion, DMF is converted to monomethyl fumarate that activates the hydroxycarboxylic acid receptor 2 (HCA2). Interestingly, neutrophils and monocytes expressed Hca2. To investigate whether the therapeutic effect of DMF in EBA is mediated by HCA2, we administered oral DMF to Hca2-deficient mice (Hca2-/-) and wild-type littermates (Hca2+/+) and induced EBA. DMF treatment ameliorated skin lesions in Hca2+/+ but not in Hca2-/- animals. These findings demonstrate that HCA2 is a molecular target of DMF treatment in EBA and suggest that HCA2 activation limits skin pathology by inhibiting the infiltration of neutrophils and monocytes into the skin.