Evaluating a Large Language Model's Ability to Answer Clinicians' Requests for Evidence Summaries.

Objective: This study investigated the performance of a generative artificial intelligence (AI) tool using GPT-4 in answering clinical questions in comparison with medical librarians' gold-standard evidence syntheses. Methods: Questions were extracted from an in-house database of clinical evidence requests previously answered by medical librarians. Questions with multiple parts were subdivided into individual topics. A standardized prompt was developed using the COSTAR framework. Librarians submitted each question into aiChat, an internally-managed chat tool using GPT-4, and recorded the responses. The summaries generated by aiChat were evaluated on whether they contained the critical elements used in the established gold-standard summary of the librarian. A subset of questions was randomly selected for verification of references provided by aiChat. Results: Of the 216 evaluated questions, aiChat's response was assessed as "correct" for 180 (83.3%) questions, "partially correct" for 35 (16.2%) questions, and "incorrect" for 1 (0.5%) question. No significant differences were observed in question ratings by question category (p=0.39). For a subset of 30% (n=66) of questions, 162 references were provided in the aiChat summaries, and 60 (37%) were confirmed as nonfabricated. Conclusions: Overall, the performance of a generative AI tool was promising. However, many included references could not be independently verified, and attempts were not made to assess whether any additional concepts introduced by aiChat were factually accurate. Thus, we envision this being the first of a series of investigations designed to further our understanding of how current and future versions of generative AI can be used and integrated into medical librarians' workflow.

Using a Natural Language Processing Approach to Support Rapid Knowledge Acquisition.

Implementing artificial intelligence to extract insights from large, real-world clinical data sets can supplement and enhance knowledge management efforts for health sciences research and clinical care. At Vanderbilt University Medical Center (VUMC), the in-house developed Word Cloud natural language processing system extracts coded concepts from patient records in VUMC's electronic health record repository using the Unified Medical Language System terminology. Through this process, the Word Cloud extracts the most prominent concepts found in the clinical documentation of a specific patient or population. The Word Cloud provides added value for clinical care decision-making and research. This viewpoint paper describes a use case for how the VUMC Center for Knowledge Management leverages the condition-disease associations represented by the Word Cloud to aid in the knowledge generation needed to inform the interpretation of phenome-wide association studies.

Epidemiology of Maternal Nutritional Status and Risk of Adverse Birth Outcomes in Undernourished Mothers with Sickle Cell Disease: A Systematic Review and Meta-Analysis Protocol.

In pregnancies complicated by sickle cell disease (SCD), the maternal-fetal dyad is at high risk for mortality and morbidity. In healthy pregnancies, maternal nutritional status is a critical factor for the healthy growth and development of the fetus. However, there are no reviews of the current research on the nutritional status of pregnant women with SCD and pregnancy outcomes. First, we aim to assess the burden of malnutrition in pregnant women with SCD. Next, we aim to systematically evaluate if pregnant women with SCD who have poor nutritional status are at increased risk for adverse birth outcomes compared to pregnant women with sickle cell disease and normal nutritional status. We will systematically search multiple electronic databases. Our exposure is pregnant women with SCD and poor nutritional status. The primary outcomes of interest include low birth weight (categorical) and birth weight z-scores (continuous). We will also evaluate maternal and perinatal outcomes as secondary outcomes. We will evaluate the risk of bias and overall certainty of evidence with Risk of Bias in Non-randomized Studies-of Interventions (ROBINS-I), and the overall evidence will be assessed using Grading of Recommendation Assessment, Development, and Evaluation (GRADE) criteria. We will pool findings with a meta-analysis if sufficient homogeneity exists among studies. Findings will be published in a peer-reviewed journal and disseminated to SCD advocacy groups. PROSPERO registration number: 429412.

Staff preparedness is key to successful change in the NHS.

Annette Chowthi-Williams, Senior Lecturer, University of Roehampton (A.Chowthi-Williams@roehampton.ac.uk).

Systematic review of international studies evaluating MDRD and CKD-EPI estimated glomerular filtration rate (eGFR) equations in Black adults.

Use of race adjustment in estimating glomerular filtration rate (eGFR) has been challenged given concerns that it may negatively impact the clinical care of Black patients, as it results in Black patients being systematically assigned higher eGFR values than non-Black patients. We conducted a systematic review to assess how well eGFR, with and without race adjustment, estimates measured GFR (mGFR) in Black adults globally. A search across multiple databases for articles published from 1999 to May 2021 that compared eGFR to mGFR and reported outcomes by Black race was performed. We included studies that assessed eGFR using the Modification of Diet in Renal Disease (MDRD) and Chronic Kidney Disease Epidemiology Collaboration (CKD-EPICr) creatinine equations. Risk of study bias and applicability were assessed with the QUality Assessment of Diagnostic Accuracy Studies-2. Of 13,167 citations identified, 12 met the data synthesis criteria (unique patient cohorts in which eGFR was compared to mGFR with and without race adjustment). The studies included patients with and without kidney disease from Africa (n = 6), the United States (n = 3), Europe (n = 2), and Brazil (n = 1). Of 11 CKD-EPI equation studies, all assessed bias, 8 assessed accuracy, 6 assessed precision, and 5 assessed correlation/concordance. Of 7 MDRD equation studies, all assessed bias, 6 assessed accuracy, 5 assessed precision, and 3 assessed correlation/concordance. The majority of studies found that removal of race adjustment improved bias, accuracy, and precision of eGFR equations for Black adults. Risk of study bias was often unclear, but applicability concerns were low. Our systematic review supports the need for future studies to be conducted in diverse populations to assess the possibility of alternative approaches for estimating GFR. This study additionally provides systematic-level evidence for the American Society of Nephrology-National Kidney Foundation Task Force efforts to pursue other options for GFR estimation.

Using SPI-Hub™ to Promote the Key Role of Prepublishing in Healthcare.

With the need to quickly advance knowledge dissemination in rapid-paced fields, and more recently in response to the urgency of the COVID-19 pandemic, prepublishing has been brought to the forefront. SPI-Hub™, a publicly available journal selection decision support tool, is being strategically enhanced to address prospective authors' critical needs in navigating and selecting the most appropriate preprint or traditional publication venue.

A narrative review of the impact of the transition to ICD-10 and ICD-10-CM/PCS.

OBJECTIVES: The United States transitioned to the tenth version of the International Classification of Diseases (ICD) system (ICD-10) for mortality coding in 1999 and to the International Classification of Diseases, Clinical Modification and Procedure Coding System (ICD-10-CM/PCS) on October 1, 2015. The purpose of this study was to conduct a narrative literature review to better understand the impact of the implementation of ICD-10/ICD-10-CM/PCS. MATERIALS AND METHODS: We searched English-language articles in PubMed, Web of Science, and Business Source Complete and reviewed websites of relevant professional associations, government agencies, research groups, and ICD-10 news aggregators to identify literature on the impact of the ICD-10/ICD-10-CM/PCS transition. We used Google to search for additional gray literature and used handsearching of the references of the most on-target articles to help ensure comprehensiveness. RESULTS: Impact areas reported in the literature include: productivity and staffing, costs, reimbursement, coding accuracy, mapping between ICD versions, morbidity and mortality surveillance, and patient care. With the exception of morbidity and mortality surveillance, quantitative studies describing the actual impact of the ICD-10/ICD-10-CM/PCS implementation were limited and much of the literature was based on the ICD-10-CM/PCS transition rather than the earlier conversion to ICD-10 for mortality coding. DISCUSSION: This study revealed several gaps in the literature that limit the ability to draw reliable conclusions about the overall impact, positive or negative, of moving to ICD-10/ICD-10-CM/PCS in the United States. CONCLUSION: These knowledge gaps present an opportunity for future research and knowledge sharing and will be important to consider when planning for ICD-11.

SPI-Hub™: a gateway to scholarly publishing information.

BACKGROUND: Advances in the health sciences rely on sharing research and data through publication. As information professionals are often asked to contribute their knowledge to assist clinicians and researchers in selecting journals for publication, the authors recognized an opportunity to build a decision support tool, SPI-Hub: Scholarly Publishing Information Hub™, to capture the team's collective publishing industry knowledge, while carefully retaining the quality of service. CASE PRESENTATION: SPI-Hub's decision support functionality relies on a data framework that describes journal publication policies and practices through a newly designed metadata structure, the Knowledge Management Journal Record™. Metadata fields are populated through a semi-automated process that uses custom programming to access content from multiple sources. Each record includes 25 metadata fields representing best publishing practices. Currently, the database includes more than 24,000 health sciences journal records. To correctly capture the resources needed for both completion and future maintenance of the project, the team conducted an internal study to assess time requirements for completing records through different stages of automation. CONCLUSIONS: The journal decision support tool, SPI-Hub, provides an opportunity to assess publication practices by compiling data from a variety of sources in a single location. Automated and semi-automated approaches have effectively reduced the time needed for data collection. Through a comprehensive knowledge management framework and the incorporation of multiple quality points specific to each journal, SPI-Hub provides prospective users with both recommendations for publication and holistic assessment of the trustworthiness of journals in which to publish research and acquire trusted knowledge.

Why equating all evidence searches to systematic reviews defies their role in information seeking.

All too often the quality and rigor of topic investigations is inaccurately conveyed to information professionals, resulting in a mischaracterization of the research, which, if left unchecked and published, may in turn mislead potential readers. Accurately understanding and categorizing the types of topic investigation searches that are requested of information professionals is critical to both meeting requestors' needs and reflecting their intended methodological approaches. Information professionals' expertise can be an invaluable resource to guide users through the investigative and publication process.

Evaluation of how a real time pre-registration health care curricula was managed through the application of a newly designed Change Management Model: A qualitative case study.

BACKGROUND: Curricula change in nurse education is of international importance. The pace of such change has been continuous and has triggered criticisms of inadequate preparation of practitioners. There are no change formulae for managing curricula change and despite a raft of change methods, globally change success remains low. A lack of a unified voice, undue focus on cognition, and arguably no existing models for academia and a literature gap contribute to change challenge. A new Change Management Model designed from research with emotion as its underpinning philosophy is evaluated. OBJECTIVE: Evaluation of a newly designed Change Management Model through a real time pre-registration health care curricula change. DESIGN: A qualitative case study was adopted. The single case study was the new pre-registration health care curricula. SETTING: This study took place in a Faculty of Health and Social care in one HEI in the UK. PARTICIPANTS: Four senior academics and fifteen academics across professions and specialisms involved in the curricula change took part in the study. RESULTS: The findings suggested that leadership operated differently throughout the organisation. Distributive and collective leadership created a critical mass of people to help deliver the new curricula but academics felt excluded at the strategic level. Emotion at the strategic level inhibited innovation but boosted engagement, emotional relationships and creativity at the operational level. Face to face communication was favoured for its emotional connection. A top down approach created an emotional disconnect and impacted inclusiveness, engagement, empowerment, vision and readiness for change. CONCLUSION: Testing the new model widely not only in organisations, practice and team changes but personal change in improving health and wellbeing could be beneficial. The continuing gap in knowledge on the link between emotion and curricula change, practice and organisational change and therapeutic value of the model also warrants further research.

Evaluation of how a curriculum change in nurse education was managed through the application of a business change management model: A qualitative case study.

BACKGROUND: Curriculum changes are a regular feature of nurse education, yet little is known about how such changes are managed. Research in this arena is yet to emerge. OBJECTIVE: Evaluation of how a curriculum change in nurse education was managed through the application of a business change management model. METHOD: A qualitative case study: the single case was the new curriculum, the Primary Care Pathway. PARTICIPANTS AND SETTING: One executive, three senior managers, two academics and nineteen students participated in this study in one faculty of health and social care in a higher education institution. RESULTS: The findings suggest that leadership was pivotal to the inception of the programme and guiding teams managed the change and did not take on a leadership role. The vision for the change and efforts to communicate it did not reach the frontline. Whilst empowerment was high amongst stakeholders and students, academics felt dis-empowered. Short-term wins were not significant in keeping up the momentum of change. The credibility of the change was under challenge and the concept of the new programme was not yet embedded in academia. CONCLUSION: Differences between the strategic and operational part of the organisation surfaced with many challenges occurring at the implementation stage. The business change model used was valuable, but was found to not be applicable during curriculum changes in nurse education. A new change model emerged, and a tool was developed alongside to aid future curriculum changes.

Using Best Practices to Extract, Organize, and Reuse Embedded Decision Support Content Knowledge Rules from Mature Clinical Systems.

Clinical decision support (CDS) knowledge, embedded over time in mature medical systems, presents an interesting and complex opportunity for information organization, maintenance, and reuse. To have a holistic view of all decision support requires an in-depth understanding of each clinical system as well as expert knowledge of the latest evidence. This approach to clinical decision support presents an opportunity to unify and externalize the knowledge within rules-based decision support. Driven by an institutional need to prioritize decision support content for migration to new clinical systems, the Center for Knowledge Management and Health Information Technology teams applied their unique expertise to extract content from individual systems, organize it through a single extensible schema, and present it for discovery and reuse through a newly created Clinical Support Knowledge Acquisition and Archival Tool (CS-KAAT). CS-KAAT can build and maintain the underlying knowledge infrastructure needed by clinical systems.

Strategically aligning a mandala of competencies to advance a transformative vision.

OBJECTIVE: This paper offers insight into the processes that have shaped the Eskind Biomedical Library's (EBL's) strategic direction and its alignment to the institution's transformative vision. SETTING: The academic biomedical library has a notable track record for developing and pioneering roles for information professionals focused on a sophisticated level of information provision that draws from and fuels practice evolutions. STRATEGY: The medical center's overall transformative vision informs the creation of a fully aligned library strategic plan designed to effectively contribute to the execution of key organizational goals. Annual goals reflect organizational priorities and contain quantifiable and measurable deliverables. Two strategic themes, facilitating genetic literacy and preserving community history, are described in detail to illustrate the concept of goal setting. CONCLUSION: The strategic planning model reflects EBL's adaptation to the ever-changing needs of its organization. The paper provides a characterization of a workable model that can be replicated by other institutions.

Integrating best evidence into patient care: a process facilitated by a seamless integration with informatics tools.

The Vanderbilt University paper discusses how the Eskind Biomedical Library at Vanderbilt University Medical Center transitioned from a simplistic approach that linked resources to the institutional electronic medical record system, StarPanel, to a value-added service that is designed to deliver highly relevant information. Clinical teams formulate complex patient-specific questions via an evidence-based medicine literature request basket linked to individual patient records. The paper transitions into discussing how the StarPanel approach acted as a springboard for two additional projects that use highly trained knowledge management librarians with informatics expertise to integrate evidence into both order sets and a patient portal, MyHealth@Vanderbilt.

Perianesthesia care of the Alzheimer's patient.

Surgery can be a confusing and traumatizing experience for the patient with Alzheimer's Disease (AD). The perianesthesia nurse can better care for the AD patient by acquiring knowledge of the symptoms and stages of the disease, communication skills specific to AD patients, and techniques useful in dealing with AD behaviors. An understanding of AD factors related to anesthesia administration, pain and pain management, as well as general post-operative care, is necessary to create a plan of care that focuses on the AD patient's unique needs. It is important that there be thorough communication along the continuum of care. The health care professional, using well-established nursing skills, paired with Alzheimer's Disease educational resources, can make the challenge of caring for the AD surgical patient less stressful and, hopefully, more satisfying for both the AD patient and the nurse.

Intermethod reliability of real-time versus delayed videotaped evaluation of a high-fidelity medical simulation septic shock scenario.

OBJECTIVES: High-fidelity medical simulation (HFMS) is increasingly utilized in resident education and evaluation. No criterion standard of assessing performance currently exists. This study compared the intermethod reliability of real-time versus videotaped evaluation of HFMS participant performance. METHODS: Twenty-five emergency medicine residents and one transitional resident participated in a septic shock HFMS scenario. Four evaluators assessed the performance of participants on technical (26-item yes/no completion) and nontechnical (seven item, five-point Likert scale assessment) scorecards. Two evaluators provided assessment in real time, and two provided delayed videotape review. After 13 scenarios, evaluators crossed over and completed the scenarios in the opposite method. Real-time evaluations were completed immediately at the end of the simulation; videotape reviewers were allowed to review the scenarios with no time limit. Agreement between raters was tested using the intraclass correlation coefficient (ICC), with Cronbach's alpha used to measure consistency among items on the scores on the checklists. RESULTS: Bland-Altman plot analysis of both conditions revealed substantial agreement between the real-time and videotaped review scores by reviewers. The mean difference between the reviewers was 0.0 (95% confidence interval [CI] = -3.7 to 3.6) on the technical evaluation and -1.6 (95% CI = -11.4 to 8.2) on the nontechnical scorecard assessment. Comparison of evaluations for the videotape technical scorecard demonstrated a Cronbach's alpha of 0.914, with an ICC of 0.842 (95% CI = 0.679 to 0.926), and the real-time technical scorecard demonstrated a Cronbach's alpha of 0.899, with an ICC of 0.817 (95% CI = 0.633 to 0.914), demonstrating excellent intermethod reliability. Comparison of evaluations for the videotape nontechnical scorecard demonstrated a Cronbach's alpha of 0.888, with an ICC of 0.798 (95% CI = 0.600 to 0.904), and the real-time nontechnical scorecard demonstrated a Cronbach's alpha of 0.833, with an ICC of 0.714 (95% CI = 0.457 to 0.861), demonstrating substantial interrater reliability. The raters were consistent in agreement on performance within each level of training, as the analysis of variance demonstrated no significant differences between the technical scorecard (p = 0.176) and nontechnical scorecard (p = 0.367). CONCLUSIONS: Real-time and videotaped-based evaluations of resident performance of both technical and nontechnical skills during an HFMS septic shock scenario provided equally reliable methods of assessment.

Highly tractable, sub-nanomolar non-steroidal glucocorticoid receptor agonists.

Starting from a non-steroidal glucocorticoid agonist aryl pyrazole derivative, the NFkappaB agonist activity was optimised in an iterative process from pIC(50) 7.5 (for 7), to pIC(50) 10.1 (for 38E1). An explanation for the SAR observed based is presented along with a proposed docking of 38E1 into the active site of the glucocorticoid receptor.

Emergency Department education improves patient knowledge of coronary artery disease risk factors but not the accuracy of their own risk perception.

BACKGROUND: Coronary artery disease (CAD) is the single largest killer of both males and females in the United States. The Emergency Department (ED) represents a unique environment in which patient education may improve coronary artery disease risk factor knowledge and relative risk perception. ED patients' understanding of CAD risk factors is often limited. Patients' perception of their own risk is often not a reflection of their true risk. We hypothesized that an American Heart Association educational video intervention would improve patients' knowledge of coronary artery disease risk factors and personal risk awareness in the ED setting. METHODS: IRB approval was obtained for this prospective observational cohort study. Our trial included 100 adult patients (age 18 and over), both male and female, using the ED population at an inner city tertiary care Level I trauma center hospital as our source of participants. Recruitment of patients began in January 2002 and ended in May 2004. RESULTS: Patients who watched the educational video did improve their knowledge of cardiac risk factors significantly when compared to patients who received no educational video intervention. In our study, this information was not retained at 30-day follow-up. However, there was still significant improvement in their knowledge when compared to baseline scores pre-intervention. Patients overestimated their risk when compared to an objective measure of risk. In both the study and control groups, patients significantly overestimated their risk pre-educational intervention, immediately post-educational intervention, and at 30-day follow-up when compared to an objective measure of risk. CONCLUSIONS: Simple educational intervention at a teachable moment (i.e. when a patient is experiencing chest pain in the ED) significantly improves patient's knowledge of CAD risk factors immediately post-intervention. This improvement in knowledge is not fully retained at 30-day follow-up, which suggests that patients may benefit from further educational intervention prior to 1 month follow-up. Patients overestimate their risk when compared to an objective measure of risk, regardless of whether they receive an educational intervention or not.

High fidelity medical simulation in the difficult environment of a helicopter: feasibility, self-efficacy and cost.

BACKGROUND: This study assessed the feasibility, self-efficacy and cost of providing a high fidelity medical simulation experience in the difficult environment of an air ambulance helicopter. METHODS: Seven of 12 EM residents in their first postgraduate year participated in an EMS flight simulation as the flight physician. The simulation used the Laerdal SimMantrade mark to present a cardiac and a trauma case in an EMS helicopter while running at flight idle. Before and after the simulation, subjects completed visual analog scales and a semi-structured interview to measure their self-efficacy, i.e. comfort with their ability to treat patients in the helicopter, and recognition of obstacles to care in the helicopter environment. After all 12 residents had completed their first non-simulated flight as the flight physician; they were surveyed about self-assessed comfort and perceived value of the simulation. Continuous data were compared between pre- and post-simulation using a paired samples t-test, and between residents participating in the simulation and those who did not using an independent samples t-test. Categorical data were compared using Fisher's exact test. Cost data for the simulation experience were estimated by the investigators. RESULTS: The simulations functioned correctly 5 out of 7 times; suggesting some refinement is necessary. Cost data indicated a monetary cost of 440 dollars and a time cost of 22 hours of skilled instructor time. The simulation and non-simulation groups were similar in their demographics and pre-hospital experiences. The simulation did not improve residents' self-assessed comfort prior to their first flight (p > 0.234), but did improve understanding of the obstacles to patient care in the helicopter (p = 0.029). Every resident undertaking the simulation agreed it was educational and it should be included in their training. Qualitative data suggested residents would benefit from high fidelity simulation in other environments, including ground transport and for running codes in hospital. CONCLUSION: It is feasible to provide a high fidelity medical simulation experience in the difficult environment of the air ambulance helicopter, although further experience is necessary to eliminate practical problems. Simulation improves recognition of the challenges present and provides an important opportunity for training in challenging environments. However, use of simulation technology is expensive both in terms of monetary outlay and of personnel involvement. The benefits of this technology must be weighed against the cost for each institution.