Harnessing PubMed User Query Logs for Post Hoc Explanations of Recommended Similar Articles.

Searching for a related article based on a reference article is an integral part of scientific research. PubMed, like many academic search engines, has a "similar articles" feature that recommends articles relevant to the current article viewed by a user. Explaining recommended items can be of great utility to users, particularly in the literature search process. With more than a million biomedical papers being published each year, explaining the recommended similar articles would facilitate researchers and clinicians in searching for related articles. Nonetheless, the majority of current literature recommendation systems lack explanations for their suggestions. We employ a post hoc approach to explaining recommendations by identifying relevant tokens in the titles of similar articles. Our major contribution is building PubCLogs by repurposing 5.6 million pairs of coclicked articles from PubMed's user query logs. Using our PubCLogs dataset, we train the Highlight Similar Article Title (HSAT), a transformer-based model designed to select the most relevant parts of the title of a similar article, based on the title and abstract of a seed article. HSAT demonstrates strong performance in our empirical evaluations, achieving an F1 score of 91.72 percent on the PubCLogs test set, considerably outperforming several baselines including BM25 (70.62), MPNet (67.11), MedCPT (62.22), GPT-3.5 (46.00), and GPT-4 (64.89). Additional evaluations on a separate, manually annotated test set further verifies HSAT's performance. Moreover, participants of our user study indicate a preference for HSAT, due to its superior balance between conciseness and comprehensiveness. Our study suggests that repurposing user query logs of academic search engines can be a promising way to train state-of-the-art models for explaining literature recommendation.

Leveraging Professional Radiologists' Expertise to Enhance LLMs' Evaluation for Radiology Reports.

In radiology, Artificial Intelligence (AI) has significantly advanced report generation, but automatic evaluation of these AI-produced reports remains challenging. Current metrics, such as Conventional Natural Language Generation (NLG) and Clinical Efficacy (CE), often fall short in capturing the semantic intricacies of clinical contexts or overemphasize clinical details, undermining report clarity. To overcome these issues, our proposed method synergizes the expertise of professional radiologists with Large Language Models (LLMs), like GPT-3.5 and GPT-4. Utilizing In-Context Instruction Learning (ICIL) and Chain of Thought (CoT) reasoning, our approach aligns LLM evaluations with radiologist standards, enabling detailed comparisons between human and AI-generated reports. This is further enhanced by a Regression model that aggregates sentence evaluation scores. Experimental results show that our "Detailed GPT-4 (5-shot)" model achieves a 0.48 score, outperforming the METEOR metric by 0.19, while our "Regressed GPT-4" model shows even greater alignment with expert evaluations, exceeding the best existing metric by a 0.35 margin. Moreover, the robustness of our explanations has been validated through a thorough iterative strategy. We plan to publicly release annotations from radiology experts, setting a new standard for accuracy in future assessments. This underscores the potential of our approach in enhancing the quality assessment of AI-driven medical reports.

The negative spillover impact of the COVID-19 pandemic on Hong Kong diabetes patients' mortality and cardiovascular disease.

BACKGROUND: The spillover impact from disrupted healthcare services for non-COVID-infected diabetes mellitus (DM) patients caused by the reshuffling of the manpower during the pandemic remains understudied, especially in Hong Kong where healthcare resources were already strained before the pandemic. AIM: To evaluate the spill-over effect of the Pandemic on Hong Kong diabetes patients, we examined the change in all-cause mortality and the incidence of cardiovascular disease (CVD) from 2012 to 2021. METHOD: This retrospective cohort study analyzed data from Hong Kong Hospital Authority healthcare records covering all publicly provided care. Adults diagnosed with DM on/before December 31, 2010, without CVD before January 2012 were included. The 2016-2019 average all-cause mortality and the incidence of CVD after age-standardization represented the pre-pandemic levels. Subjects would leave the cohort after being infected with COVID-19. RESULTS: A cohort of 159,693 patients with diabetes was identified and followed up for 10 years from January 2012 to December 2021. Compared to the pre-pandemic levels, 2020 saw a 12% increase in age-standardized mortality per 10,000 diabetic patients (incidence rate ratio [95% CI]: 1.12 [1.10 - 1.14]), but no significant change in age-standardized CVD incidence. However, in 2021, there were 11% (1.11[1.10 - 1.13]) and 13% (1.13 [1.11 - 1.15]) more new CVD cases and deaths, respectively, versus the pre-pandemic period. CONCLUSION: The COVID-19 outbreak in 2020 had negative spillover impacts on DM patients without COVID-19 in Hong Kong, with a higher mortality in 2020 and 2021 compared with the pre-pandemic level.

GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain Databases.

Gene set knowledge discovery is essential for advancing human functional genomics. Recent studies have shown promising performance by harnessing the power of Large Language Models (LLMs) on this task. Nonetheless, their results are subject to several limitations common in LLMs such as hallucinations. In response, we present GeneAgent, a first-of-its-kind language agent featuring self-verification capability. It autonomously interacts with various biological databases and leverages relevant domain knowledge to improve accuracy and reduce hallucination occurrences. Benchmarking on 1,106 gene sets from different sources, GeneAgent consistently outperforms standard GPT-4 by a significant margin. Moreover, a detailed manual review confirms the effectiveness of the self-verification module in minimizing hallucinations and generating more reliable analytical narratives. To demonstrate its practical utility, we apply GeneAgent to seven novel gene sets derived from mouse B2905 melanoma cell lines, with expert evaluations showing that GeneAgent offers novel insights into gene functions and subsequently expedites knowledge discovery.

An Organophosphorescence Probe with Ultralong Lifetime and Intrinsic Tissue Selectivity for Specific Tumor Imaging and Guided Tumor Surgery.

Organic phosphorescent materials are excellent candidates for use in tumor imaging. However, a systematic comparison of the effects of the intensity, lifetime, and wavelength of phosphorescent emissions on bioimaging performance has not yet been undertaken. In addition, there have been few reports on organic phosphorescent materials that specifically distinguish tumors from normal tissues. This study addresses these gaps and reveals that longer lifetimes effectively increase the signal intensity, whereas longer wavelengths enhance the penetration depth. Conversely, a strong emission intensity with a short lifetime does not necessarily yield robust imaging signals. Building upon these findings, an organo-phosphorescent material with a lifetime of 0.94 s was designed for tumor imaging. Remarkably, the phosphorescent signals of various organic nanoparticles are nearly extinguished in blood-rich organs because of the quenching effect of iron ions. Moreover, for the first time, we demonstrated that iron ions universally quench the phosphorescence of organic room-temperature phosphorescent materials, which is an inherent property of such substances. Leveraging this property, both the normal liver and hepatitis tissues exhibit negligible phosphorescent signals, whereas liver tumors display intense phosphorescence. Therefore, phosphorescent materials, unlike chemiluminescent or fluorescent materials, can exploit this unique inherent property to selectively distinguish liver tumor tissues from normal tissues without additional modifications or treatments.

Associations and attributable burden between risk factors and all-cause and cause-specific mortality at different ages in patients with hypertension.

It remains unclear the age-specific associations of risk factors with deaths and mortality burden attributable across age. In a territory-wide retrospective cohort, 1,012,228 adults with hypertension were identified. Comorbidities including diabetes, chronic kidney disease (CKD), cardiovascular disease (CVD), heart failure, and cancer, and risk factors including current smoking and suboptimal control of blood pressure (BP), glucose and low-density lipoprotein cholesterol were defined. Associations of comorbidities/risk factors with all-cause and cause-specific mortality across age groups (18-54, 55-64, 65-74, and ≥75 years) were assessed. Population attributable fractions were also quantified. During a median follow-up of 10.7 years, 244,268 (24.1%) patients died, with pneumonia (7.2%), cancer (5.1%), and CVD (4.2%) being the leading causes. Despite increasing deaths with age, relative risk of mortality related to comorbidities/risk factors decreased with age; similar patterns were found for cause-specific mortality. The assessed risk factors accounted for 24.0% (95% CI 22.5%, 25.4%) deaths, with highest proportion in the youngest group (33.5% [28.1%, 38.5%] in 18-54 years vs 19.4% [17.0%, 21.6%] in ≥75 years). For mortality burden, CKD was the overall leading risk factor (12.7% [12.4%, 12.9%]) with higher proportions in older patients (11.1-13.1% in ≥65 years), while diabetes was the leading risk factor in younger patients (15.9-13.5% in 18-54 years). The association of comorbidities or risk factors with mortality is stronger in younger patients with hypertension, despite lower absolute mortality in young patients than in the elderly. Leading risk factors differed across age, highlighting the importance of targeted and precise risk management.

Endowing Pt(IV) with Perfluorocarbon Chains and Human Serum Albumin Encapsulation for Highly Effective Antitumor Chemoimmunotherapy.

Tumor metastases and reoccurrence are considered the leading causes of cancer-associated deaths. As an emerging therapeutic method, increasing research efforts have been devoted to immunogenic cell death (ICD)-inducing compounds to solve the challenge. The clinically approved chemotherapeutic Pt complexes are not or are only poorly able to trigger ICD. Herein, the axial functionalization of the Pt(II) complex cisplatin with perfluorocarbon chains into ICD-inducing Pt(IV) prodrugs is reported. Strikingly, while the Pt(II) complex as well as the perfluorocarbon ligands did not induce ICD, the Pt(IV) prodrug demonstrated unexpectantly the induction of ICD through accumulation in the endoplasmic reticulum and generation of reactive oxygen species in this organelle. To enhance the pharmacological properties, the compound was encapsulated with human serum albumin into nanoparticles. While selectively accumulating in the tumorous tissue, the nanoparticles demonstrated a strong tumor growth inhibitory effect against osteosarcoma inside a mouse model. In vivo tumor vaccine analysis also demonstrated the ability of Pt(IV) to be an ideal ICD inducer. Overall, this study reports on axially perfluorocarbon chain-modified Pt(IV) complexes for ICD induction and chemoimmunotherapy in osteosarcoma.

Leveraging generative AI for clinical evidence synthesis needs to ensure trustworthiness.

Evidence-based medicine promises to improve the quality of healthcare by empowering medical decisions and practices with the best available evidence. The rapid growth of medical evidence, which can be obtained from various sources, poses a challenge in collecting, appraising, and synthesizing the evidential information. Recent advancements in generative AI, exemplified by large language models, hold promise in facilitating the arduous task. However, developing accountable, fair, and inclusive models remains a complicated undertaking. In this perspective, we discuss the trustworthiness of generative AI in the context of automated summarization of medical evidence.

PubTator 3.0: an AI-powered literature resource for unlocking biomedical knowledge.

PubTator 3.0 (https://www.ncbi.nlm.nih.gov/research/pubtator3/) is a biomedical literature resource using state-of-the-art AI techniques to offer semantic and relation searches for key concepts like proteins, genetic variants, diseases and chemicals. It currently provides over one billion entity and relation annotations across approximately 36 million PubMed abstracts and 6 million full-text articles from the PMC open access subset, updated weekly. PubTator 3.0's online interface and API utilize these precomputed entity relations and synonyms to provide advanced search capabilities and enable large-scale analyses, streamlining many complex information needs. We showcase the retrieval quality of PubTator 3.0 using a series of entity pair queries, demonstrating that PubTator 3.0 retrieves a greater number of articles than either PubMed or Google Scholar, with higher precision in the top 20 results. We further show that integrating ChatGPT (GPT-4) with PubTator APIs dramatically improves the factuality and verifiability of its responses. In summary, PubTator 3.0 offers a comprehensive set of features and tools that allow researchers to navigate the ever-expanding wealth of biomedical literature, expediting research and unlocking valuable insights for scientific discovery.

Quality of Answers of Generative Large Language Models Versus Peer Users for Interpreting Laboratory Test Results for Lay Patients: Evaluation Study.

BACKGROUND: Although patients have easy access to their electronic health records and laboratory test result data through patient portals, laboratory test results are often confusing and hard to understand. Many patients turn to web-based forums or question-and-answer (Q&A) sites to seek advice from their peers. The quality of answers from social Q&A sites on health-related questions varies significantly, and not all responses are accurate or reliable. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to have their questions answered. OBJECTIVE: We aimed to assess the feasibility of using LLMs to generate relevant, accurate, helpful, and unharmful responses to laboratory test-related questions asked by patients and identify potential issues that can be mitigated using augmentation approaches. METHODS: We collected laboratory test result-related Q&A data from Yahoo! Answers and selected 53 Q&A pairs for this study. Using the LangChain framework and ChatGPT web portal, we generated responses to the 53 questions from 5 LLMs: GPT-4, GPT-3.5, LLaMA 2, MedAlpaca, and ORCA_mini. We assessed the similarity of their answers using standard Q&A similarity-based evaluation metrics, including Recall-Oriented Understudy for Gisting Evaluation, Bilingual Evaluation Understudy, Metric for Evaluation of Translation With Explicit Ordering, and Bidirectional Encoder Representations from Transformers Score. We used an LLM-based evaluator to judge whether a target model had higher quality in terms of relevance, correctness, helpfulness, and safety than the baseline model. We performed a manual evaluation with medical experts for all the responses to 7 selected questions on the same 4 aspects. RESULTS: Regarding the similarity of the responses from 4 LLMs; the GPT-4 output was used as the reference answer, the responses from GPT-3.5 were the most similar, followed by those from LLaMA 2, ORCA_mini, and MedAlpaca. Human answers from Yahoo data were scored the lowest and, thus, as the least similar to GPT-4-generated answers. The results of the win rate and medical expert evaluation both showed that GPT-4's responses achieved better scores than all the other LLM responses and human responses on all 4 aspects (relevance, correctness, helpfulness, and safety). LLM responses occasionally also suffered from lack of interpretation in one's medical context, incorrect statements, and lack of references. CONCLUSIONS: By evaluating LLMs in generating responses to patients' laboratory test result-related questions, we found that, compared to other 4 LLMs and human answers from a Q&A website, GPT-4's responses were more accurate, helpful, relevant, and safer. There were cases in which GPT-4 responses were inaccurate and not individualized. We identified a number of ways to improve the quality of LLM responses, including prompt engineering, prompt augmentation, retrieval-augmented generation, and response evaluation.

Quinic acid regulated TMA/TMAO-related lipid metabolism and vascular endothelial function through gut microbiota to inhibit atherosclerotic.

BACKGROUND: Quinic acid (QA) and its derivatives have good lipid-lowering and hepatoprotective functions, but their role in atherosclerosis remains unknown. This study attempted to investigate the mechanism of QA on atherogenesis in Apoe-/- mice induced by HFD. METHODS: HE staining and oil red O staining were used to observe the pathology. The PCSK9, Mac-3 and SM22a expressions were detected by IHC. Cholesterol, HMGB1, TIMP-1 and CXCL13 levels were measured by biochemical and ELISA. Lipid metabolism and the HMGB1-SREBP2-SR-BI pathway were detected by PCR and WB. 16 S and metabolomics were used to detect gut microbiota and serum metabolites. RESULTS: QA or low-frequency ABX inhibited weight gain and aortic tissue atherogenesis in HFD-induced Apoe-/- mice. QA inhibited the increase of cholesterol, TMA, TMAO, CXCL13, TIMP-1 and HMGB1 levels in peripheral blood of Apoe-/- mice induced by HFD. Meanwhile, QA or low-frequency ABX treatment inhibited the expression of CAV-1, ABCA1, Mac-3 and SM22α, and promoted the expression of SREBP-1 and LXR in the vascular tissues of HFD-induced Apoe-/- mice. QA reduced Streptococcus_danieliae abundance, and promoted Lactobacillus_intestinalis and Ileibacterium_valens abundance in HFD-induced Apoe-/- mice. QA altered serum galactose metabolism, promoted SREBP-2 and LDLR, inhibited IDOL, FMO3 and PCSK9 expression in liver of HFD-induced Apoe-/- mice. The combined treatment of QA and low-frequency ABX regulated microbe-related Glycoursodeoxycholic acid and GLYCOCHENODEOXYCHOLATE metabolism in HFD-induced Apoe-/- mice. QA inhibited TMAO or LDL-induced HCAECs damage and HMGB1/SREBP2 axis dysfunction, which was reversed by HMGB1 overexpression. CONCLUSIONS: QA regulated the gut-liver lipid metabolism and chronic vascular inflammation of TMA/TMAO through gut microbiota to inhibit the atherogenesis in Apoe-/- mice, and the mechanism may be related to the HMGB1/SREBP2 pathway.

Advancing entity recognition in biomedicine via instruction tuning of large language models.

MOTIVATION: Large Language Models (LLMs) have the potential to revolutionize the field of Natural Language Processing, excelling not only in text generation and reasoning tasks but also in their ability for zero/few-shot learning, swiftly adapting to new tasks with minimal fine-tuning. LLMs have also demonstrated great promise in biomedical and healthcare applications. However, when it comes to Named Entity Recognition (NER), particularly within the biomedical domain, LLMs fall short of the effectiveness exhibited by fine-tuned domain-specific models. One key reason is that NER is typically conceptualized as a sequence labeling task, whereas LLMs are optimized for text generation and reasoning tasks. RESULTS: We developed an instruction-based learning paradigm that transforms biomedical NER from a sequence labeling task into a generation task. This paradigm is end-to-end and streamlines the training and evaluation process by automatically repurposing pre-existing biomedical NER datasets. We further developed BioNER-LLaMA using the proposed paradigm with LLaMA-7B as the foundational LLM. We conducted extensive testing on BioNER-LLaMA across three widely recognized biomedical NER datasets, consisting of entities related to diseases, chemicals, and genes. The results revealed that BioNER-LLaMA consistently achieved higher F1-scores ranging from 5% to 30% compared to the few-shot learning capabilities of GPT-4 on datasets with different biomedical entities. We show that a general-domain LLM can match the performance of rigorously fine-tuned PubMedBERT models and PMC-LLaMA, biomedical-specific language model. Our findings underscore the potential of our proposed paradigm in developing general-domain LLMs that can rival SOTA performances in multi-task, multi-domain scenarios in biomedical and health applications. AVAILABILITY AND IMPLEMENTATION: Datasets and other resources are available at https://github.com/BIDS-Xu-Lab/BioNER-LLaMA.

Quality of Answers of Generative Large Language Models vs Peer Patients for Interpreting Lab Test Results for Lay Patients: Evaluation Study.

Background: Even though patients have easy access to their electronic health records and lab test results data through patient portals, lab results are often confusing and hard to understand. Many patients turn to online forums or question and answering (Q&A) sites to seek advice from their peers. However, the quality of answers from social Q&A on health-related questions varies significantly, and not all the responses are accurate or reliable. Large language models (LLMs) such as ChatGPT have opened a promising avenue for patients to get their questions answered. Objective: We aim to assess the feasibility of using LLMs to generate relevant, accurate, helpful, and unharmful responses to lab test-related questions asked by patients and to identify potential issues that can be mitigated with augmentation approaches. Methods: We first collected lab test results related question and answer data from Yahoo! Answers and selected 53 Q&A pairs for this study. Using the LangChain framework and ChatGPT web portal, we generated responses to the 53 questions from four LLMs including GPT-4, Meta LLaMA 2, MedAlpaca, and ORCA_mini. We first assessed the similarity of their answers using standard QA similarity-based evaluation metrics including ROUGE, BLEU, METEOR, BERTScore. We also utilized an LLM-based evaluator to judge whether a target model has higher quality in terms of relevance, correctness, helpfulness, and safety than the baseline model. Finally, we performed a manual evaluation with medical experts for all the responses of seven selected questions on the same four aspects. Results: Regarding the similarity of the responses from 4 LLMs, where GPT-4 output was used as the reference answer, the responses from LLaMa 2 are the most similar ones, followed by LLaMa 2, ORCA_mini, and MedAlpaca. Human answers from Yahoo data were scored lowest and thus least similar to GPT-4-generated answers. The results of Win Rate and medical expert evaluation both showed that GPT-4's responses achieved better scores than all the other LLM responses and human responses on all the four aspects (relevance, correctness, helpfulness, and safety). However, LLM responses occasionally also suffer from lack of interpretation in one's medical context, incorrect statements, and lack of references. Conclusions: By evaluating LLMs in generating responses to patients' lab test results related questions, we find that compared to other three LLMs and human answer from the Q&A website, GPT-4's responses are more accurate, helpful, relevant, and safer. However, there are cases that GPT-4 responses are inaccurate and not individualized. We identified a number of ways to improve the quality of LLM responses including prompt engineering, prompt augmentation, retrieval augmented generation, and response evaluation.

Onset of acute severe autoimmune hepatitis after severe acute respiratory syndrome coronavirus 2 infection: a case report.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection can trigger autoimmune inflammation in the liver, leading to acute autoimmune hepatitis (AIH). We herein report a case involving a 39-year-old woman with a 23-day history of yellow skin and urine. Using the revised original scoring system of the International AIH Group, we definitively diagnosed the patient with acute severe AIH (AS-AIH). She began treatment with 80 mg/day intravenous methylprednisolone, which was gradually reduced and followed by eventual transition to oral methylprednisolone. The patient finally achieved a biochemical response after 30 days of therapy, and liver transplantation was avoided. Clinicians should be aware that the onset of AS-AIH after SARS-CoV-2 infection differs from the onset of conventional AIH with respect to its clinical and pathological features. Early diagnosis and timely glucocorticoid treatment are crucial in improving outcomes.

ROS-responsive & scavenging NO nanomedicine for vascular diseases treatment by inhibiting endoplasmic reticulum stress and improving NO bioavailability.

Vascular diseases seriously threaten human life and health. Exogenous delivery of nitric oxide (NO) represents an effective approach for maintaining vascular homeostasis during pathological events. However, the overproduction of reactive oxygen species (ROS) at vascular injury sites would react with NO to produce damaging peroxynitrite (ONOO-) species and limit the therapeutic effect of NO. Hence, we design a ROS-responsive NO nanomedicine (t-PBA&NO NP) with ROS scavenging ability to solve the dilemma of NO-based therapy. t-PBA&NO NP targets NO and anti-oxidant ethyl caffeate (ECA) to the injury sites via collagen IV homing peptide. The ROS-triggered ROS depletion and ECA release potently alleviate local oxidative stress via ROS scavenging, endoplasmic reticulum and mitochondrial regulation. It subsequently maximizes vascular modulation effects of NO, without production of harmful compounds, reactive nitrogen species (RNS). Therefore, it significantly increases competitiveness of human umbilical vein endothelial cells (HUVECs) over human aortic smooth muscle cells (HASMCs) both in vitro and in vivo. The strategy proved effective in inducing faster re-endothelialization, inhibiting neointimal formation and restoring vascular homeostasis. The synergy between ROS depletion and NO therapy served as a new inspiration for the treatment of cardiovascular diseases and other ROS-associated illnesses.

Transcriptomic Analyses and Experimental Validation Identified Immune-Related lncRNA-mRNA Pair MIR210HG-BPIFC Regulating the Progression of Hypertrophic Cardiomyopathy.

Hypertrophic cardiomyopathy (HCM) is a disease in which the myocardium of the heart becomes asymmetrically thickened, malformed, disordered, and loses its normal structure and function. Recent studies have demonstrated the significant involvement of inflammatory responses in HCM. However, the precise role of immune-related long non-coding RNAs (lncRNAs) in the pathogenesis of HCM remains unclear. In this study, we performed a comprehensive analysis of immune-related lncRNAs in HCM. First, transcriptomic RNA-Seq data from both HCM patients and healthy individuals (GSE180313) were reanalyzed thoroughly. Key HCM-related modules were identified using weighted gene co-expression network analysis (WGCNA). A screening for immune-related lncRNAs was conducted within the key modules using immune-related mRNA co-expression analysis. Based on lncRNA-mRNA pairs that exhibit shared regulatory microRNAs (miRNAs), we constructed a competing endogenous RNA (ceRNA) network, comprising 9 lncRNAs and 17 mRNAs that were significantly correlated. Among the 26 lncRNA-mRNA pairs, only the MIR210HG-BPIFC pair was verified by another HCM dataset (GSE130036) and the isoprenaline (ISO)-induced HCM cell model. Furthermore, knockdown of MIR210HG increased the regulatory miRNAs and decreased the mRNA expression of BPIFC correspondingly in AC16 cells. Additionally, the analysis of immune cell infiltration indicated that the MIR210HG-BPIFC pair was potentially involved in the infiltration of naïve CD4+ T cells and CD8+ T cells. Together, our findings indicate that the decreased expression of the lncRNA-mRNA pair MIR210HG-BPIFC was significantly correlated with the pathogenesis of the disease and may be involved in the immune cell infiltration in the mechanism of HCM.

GeneGPT: augmenting large language models with domain tools for improved access to biomedical information.

MOTIVATION: While large language models (LLMs) have been successfully applied to various tasks, they still face challenges with hallucinations. Augmenting LLMs with domain-specific tools such as database utilities can facilitate easier and more precise access to specialized knowledge. In this article, we present GeneGPT, a novel method for teaching LLMs to use the Web APIs of the National Center for Biotechnology Information (NCBI) for answering genomics questions. Specifically, we prompt Codex to solve the GeneTuring tests with NCBI Web APIs by in-context learning and an augmented decoding algorithm that can detect and execute API calls. RESULTS: Experimental results show that GeneGPT achieves state-of-the-art performance on eight tasks in the GeneTuring benchmark with an average score of 0.83, largely surpassing retrieval-augmented LLMs such as the new Bing (0.44), biomedical LLMs such as BioMedLM (0.08) and BioGPT (0.04), as well as GPT-3 (0.16) and ChatGPT (0.12). Our further analyses suggest that: First, API demonstrations have good cross-task generalizability and are more useful than documentations for in-context learning; second, GeneGPT can generalize to longer chains of API calls and answer multi-hop questions in GeneHop, a novel dataset introduced in this work; finally, different types of errors are enriched in different tasks, providing valuable insights for future improvements. AVAILABILITY AND IMPLEMENTATION: The GeneGPT code and data are publicly available at https://github.com/ncbi/GeneGPT.

PubMed and beyond: biomedical literature search in the age of artificial intelligence.

Biomedical research yields vast information, much of which is only accessible through the literature. Consequently, literature search is crucial for healthcare and biomedicine. Recent improvements in artificial intelligence (AI) have expanded functionality beyond keywords, but they might be unfamiliar to clinicians and researchers. In response, we present an overview of over 30 literature search tools tailored to common biomedical use cases, aiming at helping readers efficiently fulfill their information needs. We first discuss recent improvements and continued challenges of the widely used PubMed. Then, we describe AI-based literature search tools catering to five specific information needs: 1. Evidence-based medicine. 2. Precision medicine and genomics. 3. Searching by meaning, including questions. 4. Finding related articles with literature recommendation. 5. Discovering hidden associations through literature mining. Finally, we discuss the impacts of recent developments of large language models such as ChatGPT on biomedical information seeking.

PubTator 3.0: an AI-powered Literature Resource for Unlocking Biomedical Knowledge.

PubTator 3.0 (https://www.ncbi.nlm.nih.gov/research/pubtator3/) is a biomedical literature resource using state-of-the-art AI techniques to offer semantic and relation searches for key concepts like proteins, genetic variants, diseases, and chemicals. It currently provides over one billion entity and relation annotations across approximately 36 million PubMed abstracts and 6 million full-text articles from the PMC open access subset, updated weekly. PubTator 3.0's online interface and API utilize these precomputed entity relations and synonyms to provide advanced search capabilities and enable large-scale analyses, streamlining many complex information needs. We showcase the retrieval quality of PubTator 3.0 using a series of entity pair queries, demonstrating that PubTator 3.0 retrieves a greater number of articles than either PubMed or Google Scholar, with higher precision in the top 20 results. We further show that integrating ChatGPT (GPT-4) with PubTator APIs dramatically improves the factuality and verifiability of its responses. In summary, PubTator 3.0 offers a comprehensive set of features and tools that allow researchers to navigate the ever-expanding wealth of biomedical literature, expediting research and unlocking valuable insights for scientific discovery.