Evaluating a Large Language Model's Ability to Answer Clinicians' Requests for Evidence Summaries.

Objective: This study investigated the performance of a generative artificial intelligence (AI) tool using GPT-4 in answering clinical questions in comparison with medical librarians' gold-standard evidence syntheses. Methods: Questions were extracted from an in-house database of clinical evidence requests previously answered by medical librarians. Questions with multiple parts were subdivided into individual topics. A standardized prompt was developed using the COSTAR framework. Librarians submitted each question into aiChat, an internally-managed chat tool using GPT-4, and recorded the responses. The summaries generated by aiChat were evaluated on whether they contained the critical elements used in the established gold-standard summary of the librarian. A subset of questions was randomly selected for verification of references provided by aiChat. Results: Of the 216 evaluated questions, aiChat's response was assessed as "correct" for 180 (83.3%) questions, "partially correct" for 35 (16.2%) questions, and "incorrect" for 1 (0.5%) question. No significant differences were observed in question ratings by question category (p=0.39). For a subset of 30% (n=66) of questions, 162 references were provided in the aiChat summaries, and 60 (37%) were confirmed as nonfabricated. Conclusions: Overall, the performance of a generative AI tool was promising. However, many included references could not be independently verified, and attempts were not made to assess whether any additional concepts introduced by aiChat were factually accurate. Thus, we envision this being the first of a series of investigations designed to further our understanding of how current and future versions of generative AI can be used and integrated into medical librarians' workflow.

Using a Natural Language Processing Approach to Support Rapid Knowledge Acquisition.

Implementing artificial intelligence to extract insights from large, real-world clinical data sets can supplement and enhance knowledge management efforts for health sciences research and clinical care. At Vanderbilt University Medical Center (VUMC), the in-house developed Word Cloud natural language processing system extracts coded concepts from patient records in VUMC's electronic health record repository using the Unified Medical Language System terminology. Through this process, the Word Cloud extracts the most prominent concepts found in the clinical documentation of a specific patient or population. The Word Cloud provides added value for clinical care decision-making and research. This viewpoint paper describes a use case for how the VUMC Center for Knowledge Management leverages the condition-disease associations represented by the Word Cloud to aid in the knowledge generation needed to inform the interpretation of phenome-wide association studies.

Epidemiology of Maternal Nutritional Status and Risk of Adverse Birth Outcomes in Undernourished Mothers with Sickle Cell Disease: A Systematic Review and Meta-Analysis Protocol.

In pregnancies complicated by sickle cell disease (SCD), the maternal-fetal dyad is at high risk for mortality and morbidity. In healthy pregnancies, maternal nutritional status is a critical factor for the healthy growth and development of the fetus. However, there are no reviews of the current research on the nutritional status of pregnant women with SCD and pregnancy outcomes. First, we aim to assess the burden of malnutrition in pregnant women with SCD. Next, we aim to systematically evaluate if pregnant women with SCD who have poor nutritional status are at increased risk for adverse birth outcomes compared to pregnant women with sickle cell disease and normal nutritional status. We will systematically search multiple electronic databases. Our exposure is pregnant women with SCD and poor nutritional status. The primary outcomes of interest include low birth weight (categorical) and birth weight z-scores (continuous). We will also evaluate maternal and perinatal outcomes as secondary outcomes. We will evaluate the risk of bias and overall certainty of evidence with Risk of Bias in Non-randomized Studies-of Interventions (ROBINS-I), and the overall evidence will be assessed using Grading of Recommendation Assessment, Development, and Evaluation (GRADE) criteria. We will pool findings with a meta-analysis if sufficient homogeneity exists among studies. Findings will be published in a peer-reviewed journal and disseminated to SCD advocacy groups. PROSPERO registration number: 429412.

Systematic review of international studies evaluating MDRD and CKD-EPI estimated glomerular filtration rate (eGFR) equations in Black adults.

Use of race adjustment in estimating glomerular filtration rate (eGFR) has been challenged given concerns that it may negatively impact the clinical care of Black patients, as it results in Black patients being systematically assigned higher eGFR values than non-Black patients. We conducted a systematic review to assess how well eGFR, with and without race adjustment, estimates measured GFR (mGFR) in Black adults globally. A search across multiple databases for articles published from 1999 to May 2021 that compared eGFR to mGFR and reported outcomes by Black race was performed. We included studies that assessed eGFR using the Modification of Diet in Renal Disease (MDRD) and Chronic Kidney Disease Epidemiology Collaboration (CKD-EPICr) creatinine equations. Risk of study bias and applicability were assessed with the QUality Assessment of Diagnostic Accuracy Studies-2. Of 13,167 citations identified, 12 met the data synthesis criteria (unique patient cohorts in which eGFR was compared to mGFR with and without race adjustment). The studies included patients with and without kidney disease from Africa (n = 6), the United States (n = 3), Europe (n = 2), and Brazil (n = 1). Of 11 CKD-EPI equation studies, all assessed bias, 8 assessed accuracy, 6 assessed precision, and 5 assessed correlation/concordance. Of 7 MDRD equation studies, all assessed bias, 6 assessed accuracy, 5 assessed precision, and 3 assessed correlation/concordance. The majority of studies found that removal of race adjustment improved bias, accuracy, and precision of eGFR equations for Black adults. Risk of study bias was often unclear, but applicability concerns were low. Our systematic review supports the need for future studies to be conducted in diverse populations to assess the possibility of alternative approaches for estimating GFR. This study additionally provides systematic-level evidence for the American Society of Nephrology-National Kidney Foundation Task Force efforts to pursue other options for GFR estimation.

Using SPI-Hub™ to Promote the Key Role of Prepublishing in Healthcare.

With the need to quickly advance knowledge dissemination in rapid-paced fields, and more recently in response to the urgency of the COVID-19 pandemic, prepublishing has been brought to the forefront. SPI-Hub™, a publicly available journal selection decision support tool, is being strategically enhanced to address prospective authors' critical needs in navigating and selecting the most appropriate preprint or traditional publication venue.

A narrative review of the impact of the transition to ICD-10 and ICD-10-CM/PCS.

OBJECTIVES: The United States transitioned to the tenth version of the International Classification of Diseases (ICD) system (ICD-10) for mortality coding in 1999 and to the International Classification of Diseases, Clinical Modification and Procedure Coding System (ICD-10-CM/PCS) on October 1, 2015. The purpose of this study was to conduct a narrative literature review to better understand the impact of the implementation of ICD-10/ICD-10-CM/PCS. MATERIALS AND METHODS: We searched English-language articles in PubMed, Web of Science, and Business Source Complete and reviewed websites of relevant professional associations, government agencies, research groups, and ICD-10 news aggregators to identify literature on the impact of the ICD-10/ICD-10-CM/PCS transition. We used Google to search for additional gray literature and used handsearching of the references of the most on-target articles to help ensure comprehensiveness. RESULTS: Impact areas reported in the literature include: productivity and staffing, costs, reimbursement, coding accuracy, mapping between ICD versions, morbidity and mortality surveillance, and patient care. With the exception of morbidity and mortality surveillance, quantitative studies describing the actual impact of the ICD-10/ICD-10-CM/PCS implementation were limited and much of the literature was based on the ICD-10-CM/PCS transition rather than the earlier conversion to ICD-10 for mortality coding. DISCUSSION: This study revealed several gaps in the literature that limit the ability to draw reliable conclusions about the overall impact, positive or negative, of moving to ICD-10/ICD-10-CM/PCS in the United States. CONCLUSION: These knowledge gaps present an opportunity for future research and knowledge sharing and will be important to consider when planning for ICD-11.

SPI-Hub™: a gateway to scholarly publishing information.

BACKGROUND: Advances in the health sciences rely on sharing research and data through publication. As information professionals are often asked to contribute their knowledge to assist clinicians and researchers in selecting journals for publication, the authors recognized an opportunity to build a decision support tool, SPI-Hub: Scholarly Publishing Information Hub™, to capture the team's collective publishing industry knowledge, while carefully retaining the quality of service. CASE PRESENTATION: SPI-Hub's decision support functionality relies on a data framework that describes journal publication policies and practices through a newly designed metadata structure, the Knowledge Management Journal Record™. Metadata fields are populated through a semi-automated process that uses custom programming to access content from multiple sources. Each record includes 25 metadata fields representing best publishing practices. Currently, the database includes more than 24,000 health sciences journal records. To correctly capture the resources needed for both completion and future maintenance of the project, the team conducted an internal study to assess time requirements for completing records through different stages of automation. CONCLUSIONS: The journal decision support tool, SPI-Hub, provides an opportunity to assess publication practices by compiling data from a variety of sources in a single location. Automated and semi-automated approaches have effectively reduced the time needed for data collection. Through a comprehensive knowledge management framework and the incorporation of multiple quality points specific to each journal, SPI-Hub provides prospective users with both recommendations for publication and holistic assessment of the trustworthiness of journals in which to publish research and acquire trusted knowledge.

Why equating all evidence searches to systematic reviews defies their role in information seeking.

All too often the quality and rigor of topic investigations is inaccurately conveyed to information professionals, resulting in a mischaracterization of the research, which, if left unchecked and published, may in turn mislead potential readers. Accurately understanding and categorizing the types of topic investigation searches that are requested of information professionals is critical to both meeting requestors' needs and reflecting their intended methodological approaches. Information professionals' expertise can be an invaluable resource to guide users through the investigative and publication process.

Using Best Practices to Extract, Organize, and Reuse Embedded Decision Support Content Knowledge Rules from Mature Clinical Systems.

Clinical decision support (CDS) knowledge, embedded over time in mature medical systems, presents an interesting and complex opportunity for information organization, maintenance, and reuse. To have a holistic view of all decision support requires an in-depth understanding of each clinical system as well as expert knowledge of the latest evidence. This approach to clinical decision support presents an opportunity to unify and externalize the knowledge within rules-based decision support. Driven by an institutional need to prioritize decision support content for migration to new clinical systems, the Center for Knowledge Management and Health Information Technology teams applied their unique expertise to extract content from individual systems, organize it through a single extensible schema, and present it for discovery and reuse through a newly created Clinical Support Knowledge Acquisition and Archival Tool (CS-KAAT). CS-KAAT can build and maintain the underlying knowledge infrastructure needed by clinical systems.

Strategically aligning a mandala of competencies to advance a transformative vision.

OBJECTIVE: This paper offers insight into the processes that have shaped the Eskind Biomedical Library's (EBL's) strategic direction and its alignment to the institution's transformative vision. SETTING: The academic biomedical library has a notable track record for developing and pioneering roles for information professionals focused on a sophisticated level of information provision that draws from and fuels practice evolutions. STRATEGY: The medical center's overall transformative vision informs the creation of a fully aligned library strategic plan designed to effectively contribute to the execution of key organizational goals. Annual goals reflect organizational priorities and contain quantifiable and measurable deliverables. Two strategic themes, facilitating genetic literacy and preserving community history, are described in detail to illustrate the concept of goal setting. CONCLUSION: The strategic planning model reflects EBL's adaptation to the ever-changing needs of its organization. The paper provides a characterization of a workable model that can be replicated by other institutions.

Integrating best evidence into patient care: a process facilitated by a seamless integration with informatics tools.

The Vanderbilt University paper discusses how the Eskind Biomedical Library at Vanderbilt University Medical Center transitioned from a simplistic approach that linked resources to the institutional electronic medical record system, StarPanel, to a value-added service that is designed to deliver highly relevant information. Clinical teams formulate complex patient-specific questions via an evidence-based medicine literature request basket linked to individual patient records. The paper transitions into discussing how the StarPanel approach acted as a springboard for two additional projects that use highly trained knowledge management librarians with informatics expertise to integrate evidence into both order sets and a patient portal, MyHealth@Vanderbilt.

Using knowledge management practices to develop a state-of-the-art digital library.

Diffusing knowledge management practices within an organization encourages and facilitates reuse of the institution's knowledge commodity. Following knowledge management practices, the Eskind Biomedical Library (EBL) has created a Digital Library that uses a holistic approach for integration of information and skills to best represent both explicit and tacit knowledge inherent in libraries. EBL's Digital Library exemplifies a clear attempt to organize institutional knowledge in the field of librarianship, in an effort to positively impact clinical, research, and educational processes in the medical center.