The potential and pitfalls of using a large language model such as ChatGPT, GPT-4, or LLaMA as a clinical assistant.

OBJECTIVES: This study aims to evaluate the utility of large language models (LLMs) in healthcare, focusing on their applications in enhancing patient care through improved diagnostic, decision-making processes, and as ancillary tools for healthcare professionals. MATERIALS AND METHODS: We evaluated ChatGPT, GPT-4, and LLaMA in identifying patients with specific diseases using gold-labeled Electronic Health Records (EHRs) from the MIMIC-III database, covering three prevalent diseases-Chronic Obstructive Pulmonary Disease (COPD), Chronic Kidney Disease (CKD)-along with the rare condition, Primary Biliary Cirrhosis (PBC), and the hard-to-diagnose condition Cancer Cachexia. RESULTS: In patient identification, GPT-4 had near similar or better performance compared to the corresponding disease-specific Machine Learning models (F1-score ≥ 85%) on COPD, CKD, and PBC. GPT-4 excelled in the PBC use case, achieving a 4.23% higher F1-score compared to disease-specific "Traditional Machine Learning" models. ChatGPT and LLaMA3 demonstrated lower performance than GPT-4 across all diseases and almost all metrics. Few-shot prompts also help ChatGPT, GPT-4, and LLaMA3 achieve higher precision and specificity but lower sensitivity and Negative Predictive Value. DISCUSSION: The study highlights the potential and limitations of LLMs in healthcare. Issues with errors, explanatory limitations and ethical concerns like data privacy and model transparency suggest that these models would be supplementary tools in clinical settings. Future studies should improve training datasets and model designs for LLMs to gain better utility in healthcare. CONCLUSION: The study shows that LLMs have the potential to assist clinicians for tasks such as patient identification but false positives and false negatives must be mitigated before LLMs are adequate for real-world clinical assistance.

Persistent symptoms are associated with long term effects of COVID-19 among children and young people: Results from a systematic review and meta-analysis of controlled studies.

BACKGROUND: Research on the long-term impact on COVID-19 in children and young people (CYP) has been published at pace. We aimed to update and refine an earlier systematic review and meta-analysis to assess the current evidence for Post-COVID-19 Condition in CYP. METHODS: Studies from the previous systematic review were combined with studies from a systematic search from July 2021 to November 2022 (registration PROSPERO CRD42021233153). Eligible studies included CYP aged ≤19 years with confirmed or probable SARS-CoV-2 infection and symptoms persisting at least 12 weeks. FINDINGS: 55 studies (n = 1,139,299 participants) were included. Over two-hundred symptoms were associated with Post COVID-19 Condition. Gastrointestinal problems, headaches, cough and fever were among the most prevalent symptoms with rates of 50.2%, 35.6%, 34.7% and 25.8% respectively. Twenty-one symptoms from 11 studies were suitable for meta-analysis. There were significantly higher pooled estimates of proportions of symptoms for altered / loss of smell or taste, dyspnoea, fatigue, and myalgia in CYP with confirmed SARS-CoV-2 infection. Heterogeneity was high suggesting substantial variation amongst the included studies. CONCLUSIONS: Many CYP continue to experience symptoms after SARS-CoV-2 infection. Efforts to aid early identification and intervention of those most in need is warranted and the consequences of COVID-19 for CYP call for long-term follow-up.

Management of Pain and Cachexia in Pancreatic Cancer: Protocol for Two Systematic Reviews, Network Meta-Analysis, Surveys, and Focus Groups.

BACKGROUND:  Approximately 75% of people with pancreatic cancer experience pain, and >50% of them have cachexia (weakness and wasting of the body). However, there is considerable uncertainty regarding the management of these distressing symptoms. OBJECTIVE:  Our primary objectives are to compare the relative benefits and harms of different interventions for pain in people with unresectable pancreatic cancer and for prevention and treatment of cachexia due to pancreatic cancer, through systematic reviews and network meta-analysis. Our secondary objectives are to develop an evidence-based clinical care pathway to manage pain and prevent and treat cachexia in people with pancreatic cancer through surveys and focus groups involving patients, carers, and health care professionals. METHODS:  We will perform 2 systematic reviews of the literature related to pain and cachexia in people with pancreatic cancer using searches from Cochrane Library, MEDLINE, Embase, Science Citation Index, and trial registries. Two researchers will independently screen for eligibility and identify randomized controlled trials (no language or publication status restriction), comparing interventions for pain or cachexia based on full-texts for articles shortlisted during screening. We will assess risk of bias in the trials using the Cochrane risk of bias tool (version 2.0) and obtain data related to baseline prognostic characteristics, potential effect modifiers and outcome data related to overall survival, health-related quality of life, treatment-related complications, and resource utilisation. We aim to conduct network meta-analysis on outcomes with multiple treatment comparisons where possible, otherwise, meta-analysis with direct comparisons, or narrative synthesis. We will perform various subgroup and sensitivity analyses. Using information obtained from both systematic reviews, we will conduct 2 surveys: one directed to patients or carers to assess acceptability of interventions, and the other to health care professionals to assess feasibility of delivery in the National Health Service. Four mixed focus groups will be conducted to evaluate findings and foster consensus in the development of the care pathway. RESULTS:  Funding was awarded from April 2022 (NIHR202727). Both systematic review protocols were prospectively registered on PROSPERO in May 2022. Formal searches began thereafter. Approval by the University College London Research Ethics Committee (23563/001) was received in December 2022. Data collection began in January 2023; data analysis will begin in May 2023 (completion expected by October 2023). CONCLUSIONS:  This study will comprehensively encompass major interventions for management of pain in people with unresectable pancreatic cancer, and prevention and treatment of cachexia in people with pancreatic cancer. Key stakeholders will facilitate the development of an evidence-based care pathway, ensuring both acceptability and feasibility. The project ends in April 2024 and published results are expected within 12 months of completion. We aim to present the findings through patient group websites, conferences, and publications, irrespective of the findings, in a peer-reviewed journal. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/46335.