Retrieval Augmented Generation Enabled Generative Pre-Trained Transformer 4 (GPT-4) Performance for Clinical Trial Screening.

Background: Subject screening is a key aspect of all clinical trials; however, traditionally, it is a labor-intensive and error-prone task, demanding significant time and resources. With the advent of large language models (LLMs) and related technologies, a paradigm shift in natural language processing capabilities offers a promising avenue for increasing both quality and efficiency of screening efforts. This study aimed to test the Retrieval-Augmented Generation (RAG) process enabled Generative Pretrained Transformer Version 4 (GPT-4) to accurately identify and report on inclusion and exclusion criteria for a clinical trial. Methods: The Co-Operative Program for Implementation of Optimal Therapy in Heart Failure (COPILOT-HF) trial aims to recruit patients with symptomatic heart failure. As part of the screening process, a list of potentially eligible patients is created through an electronic health record (EHR) query. Currently, structured data in the EHR can only be used to determine 5 out of 6 inclusion and 5 out of 17 exclusion criteria. Trained, but non-licensed, study staff complete manual chart review to determine patient eligibility and record their assessment of the inclusion and exclusion criteria. We obtained the structured assessments completed by the study staff and clinical notes for the past two years and developed a workflow of clinical note-based question answering system powered by RAG architecture and GPT-4 that we named RECTIFIER (RAG-Enabled Clinical Trial Infrastructure for Inclusion Exclusion Review). We used notes from 100 patients as a development dataset, 282 patients as a validation dataset, and 1894 patients as a test set. An expert clinician completed a blinded review of patients' charts to answer the eligibility questions and determine the "gold standard" answers. We calculated the sensitivity, specificity, accuracy, and Matthews correlation coefficient (MCC) for each question and screening method. We also performed bootstrapping to calculate the confidence intervals for each statistic. Results: Both RECTIFIER and study staff answers closely aligned with the expert clinician answers across criteria with accuracy ranging between 97.9% and 100% (MCC 0.837 and 1) for RECTIFIER and 91.7% and 100% (MCC 0.644 and 1) for study staff. RECTIFIER performed better than study staff to determine the inclusion criteria of "symptomatic heart failure" with an accuracy of 97.9% vs 91.7% and an MCC of 0.924 vs 0.721, respectively. Overall, the sensitivity and specificity of determining eligibility for the RECTIFIER was 92.3% (CI) and 93.9% (CI), and study staff was 90.1% (CI) and 83.6% (CI), respectively. Conclusion: GPT-4 based solutions have the potential to improve efficiency and reduce costs in clinical trial screening. When incorporating new tools such as RECTIFIER, it is important to consider the potential hazards of automating the screening process and set up appropriate mitigation strategies such as final clinician review before patient engagement.

Results of a Remotely Delivered Hypertension and Lipid Program in More Than 10 000 Patients Across a Diverse Health Care Network.

Importance: Blood pressure (BP) and cholesterol control remain challenging. Remote care can deliver more effective care outside of traditional clinician-patient settings but scaling and ensuring access to care among diverse populations remains elusive. Objective: To implement and evaluate a remote hypertension and cholesterol management program across a diverse health care network. Design, Setting, and Participants: Between January 2018 and July 2021, 20 454 patients in a large integrated health network were screened; 18 444 were approached, and 10 803 were enrolled in a comprehensive remote hypertension and cholesterol program (3658 patients with hypertension, 8103 patients with cholesterol, and 958 patients with both). A total of 1266 patients requested education only without medication titration. Enrolled patients received education, home BP device integration, and medication titration. Nonlicensed navigators and pharmacists, supported by cardiovascular clinicians, coordinated care using standardized algorithms, task management and automation software, and omnichannel communication. BP and laboratory test results were actively monitored. Main Outcomes and Measures: Changes in BP and low-density lipoprotein cholesterol (LDL-C). Results: The mean (SD) age among 10 803 patients was 65 (11.4) years; 6009 participants (56%) were female; 1321 (12%) identified as Black, 1190 (11%) as Hispanic, 7758 (72%) as White, and 1727 (16%) as another or multiple races (including American Indian or Alaska Native, Asian, Native Hawaiian or Other Pacific Islander, unknown, other, and declined to respond; consolidated owing to small numbers); and 142 (11%) reported a preferred language other than English. A total of 424 482 BP readings and 139 263 laboratory reports were collected. In the hypertension program, the mean (SD) office BP prior to enrollment was 150/83 (18/10) mm Hg, and the mean (SD) home BP was 145/83 (20/12) mm Hg. For those engaged in remote medication management, the mean (SD) clinic BP 6 and 12 months after enrollment decreased by 8.7/3.8 (21.4/12.4) and 9.7/5.2 (22.2/12.6) mm Hg, respectively. In the education-only cohort, BP changed by a mean (SD) -1.5/-0.7 (23.0/11.1) and by +0.2/-1.9 (30.3/11.2) mm Hg, respectively (P < .001 for between cohort difference). In the lipids program, patients in remote medication management experienced a reduction in LDL-C by a mean (SD) 35.4 (43.1) and 37.5 (43.9) mg/dL at 6 and 12 months, respectively, while the education-only cohort experienced a mean (SD) reduction in LDL-C of 9.3 (34.3) and 10.2 (35.5) mg/dL at 6 and 12 months, respectively (P < .001). Similar rates of enrollment and reductions in BP and lipids were observed across different racial, ethnic, and primary language groups. Conclusions and Relevance: The results of this study indicate that a standardized remote BP and cholesterol management program may help optimize guideline-directed therapy at scale, reduce cardiovascular risk, and minimize the need for in-person visits among diverse populations.

I2b2-etl: Python application for importing electronic health data into the informatics for integrating biology and the bedside platform.

MOTIVATION: The i2b2 platform is used at major academic health institutions and research consortia for querying for electronic health data. However, a major obstacle for wider utilization of the platform is the complexity of data loading that entails a steep curve of learning the platform's complex data schemas. To address this problem, we have developed the i2b2-etl package that simplifies the data loading process, which will facilitate wider deployment and utilization of the platform. RESULTS: We have implemented i2b2-etl as a Python application that imports ontology and patient data using simplified input file schemas and provides inbuilt record number de-identification and data validation. We describe a real-world deployment of i2b2-etl for a population-management initiative at MassGeneral Brigham. AVAILABILITY AND IMPLEMENTATION: i2b2-etl is a free, open-source application implemented in Python available under the Mozilla 2 license. The application can be downloaded as compiled docker images. A live demo is available at https://i2b2clinical.org/demo-i2b2etl/ (username: demo, password: Etl@2021). SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

Use of automatic SQL generation interface to enhance transparency and validity of health-data analysis.

Analysis of health data typically requires development of queries using structured query language (SQL) by a data-analyst. As the SQL queries are manually created, they are prone to errors. In addition, accurate implementation of the queries depends on effective communication with clinical experts, that further makes the analysis error prone. As a potential resolution, we explore an alternative approach wherein a graphical interface that automatically generates the SQL queries is used to perform the analysis. The latter allows clinical experts to directly perform complex queries on the data, despite their unfamiliarity with SQL syntax. The interface provides an intuitive understanding of the query logic which makes the analysis transparent and comprehensible to the clinical study-staff, thereby enhancing the transparency and validity of the analysis. This study demonstrates the feasibility of using a user-friendly interface that automatically generate SQL for analysis of health data. It outlines challenges that will be useful for designing user-friendly tools to improve transparency and reproducibility of data analysis.

Workflow Automation for a Virtual Hypertension Management Program.

OBJECTIVES: Hypertension is a modifiable risk factor for numerous comorbidities and treating hypertension can greatly improve health outcomes. We sought to increase the efficiency of a virtual hypertension management program through workflow automation processes. METHODS: We developed a customer relationship management (CRM) solution at our institution for the purpose of improving processes and workflow for a virtual hypertension management program and describe here the development, implementation, and initial experience of this CRM system. RESULTS: Notable system features include task automation, patient data capture, multi-channel communication, integration with our electronic health record (EHR), and device integration (for blood pressure cuffs). In the five stages of our program (intake and eligibility screening, enrollment, device configuration/setup, medication titration, and maintenance), we describe some of the key process improvements and workflow automations that are enabled using our CRM platform, like automatic reminders to capture blood pressure data and present these data to our clinical team when ready for clinical decision making. We also describe key limitations of CRM, like balancing out-of-the-box functionality with development flexibility. Among our first group of referred patients, 76% (39/51) preferred email as their communication method, 26/51 (51%) were able to enroll electronically, and 63% of those enrolled (32/51) were able to transmit blood pressure data without phone support. CONCLUSION: A CRM platform could improve clinical processes through multiple pathways, including workflow automation, multi-channel communication, and device integration. Future work will examine the operational improvements of this health information technology solution as well as assess clinical outcomes.

Development of a Calculated Panel Reactive Antibody Web Service with Local Frequencies for Platelet Transfusion Refractoriness Risk Stratification.

BACKGROUND: Calculated panel reactive antibody (cPRA) scoring is used to assess whether platelet refractoriness is mediated by human leukocyte antigen (HLA) antibodies in the recipient. cPRA testing uses a national sample of US kidney donors to estimate the population frequency of HLA antigens, which may be different than HLA frequencies within local platelet inventories. We aimed to determine the impact on patient cPRA scores of using HLA frequencies derived from typing local platelet donations rather than national HLA frequencies. METHODS: We built an open-source web service to calculate cPRA scores based on national frequencies or custom-derived frequencies. We calculated cPRA scores for every hematopoietic stem cell transplantation (HSCT) patient at our institution based on the United Network for Organ Sharing (UNOS) frequencies and local frequencies. We compared frequencies and correlations between the calculators, segmented by gender. Finally, we put all scores into three buckets (mild, moderate, and high sensitizations) and looked at intergroup movement. RESULTS: 2531 patients that underwent HSCT at our institution had at least 1 antibody and were included in the analysis. Overall, the difference in medians between each group's UNOS cPRA and local cPRA was statistically significant, but highly correlated (UNOS vs. local total: 0.249 and 0.243, ρ = 0.994; UNOS vs. local female: 0.474 and 0.463, ρ = 0.987, UNOS vs. local male: 0.165 and 0.141, ρ = 0.996; P < 0.001 for all comparisons). The median difference between UNOS and cPRA scores for all patients was low (male: 0.014, interquartile range [IQR]: 0.004-0.029; female: 0.0013, IQR: 0.003-0.028). Placement of patients into three groups revealed little intergroup movement, with 2.96% (75/2531) of patients differentially classified. CONCLUSIONS: cPRA scores using local frequencies were modestly but significantly different than those obtained using national HLA frequencies. We released our software as open source, so other groups can calculate cPRA scores from national or custom-derived frequencies. Further investigation is needed to determine whether a local-HLA frequency approach can improve outcomes in patients who are immune-refractory to platelets.

Extending i2b2 into a framework for semantic abstraction of EHR to facilitate rapid development and portability of Health IT applications.

The wide gap between a care provider's conceptualization of electronic health record (EHR) and the structures for electronic health record (EHR) data storage and transmission, presents a multitude of obstacles for development of innovative Health IT applications. While developers model the EHR view of the clinicians at one end, they work with a different data view to construct health IT applications. Although there has been considerable progress to bridge this gap by evolution of developer friendly standards and tools for terminology mapping and data warehousing, there is a need for a simplified framework to facilitate development of interoperable applications. To this end, we propose a framework for creating a layer of semantic abstraction on the EHR and describe preliminary work on the implementation of this framework for management of hyperlipidemia and hypertension. Our goal is to facilitate the rapid development and portability of Health IT applications.

Phenotyping to Facilitate Accrual for a Cardiovascular Intervention.

BACKGROUND: The conventional approach for clinical studies is to identify a cohort of potentially eligible patients and then screen for enrollment. In an effort to reduce the cost and manual effort involved in the screening process, several studies have leveraged electronic health records (EHR) to refine cohorts to better match the eligibility criteria, which is referred to as phenotyping. We extend this approach to dynamically identify a cohort by repeating phenotyping in alternation with manual screening. METHODS: Our approach consists of multiple screen cycles. At the start of each cycle, the phenotyping algorithm is used to identify eligible patients from the EHR, creating an ordered list such that patients that are most likely eligible are listed first. This list is then manually screened, and the results are analyzed to improve the phenotyping for the next cycle. We describe the preliminary results and challenges in the implementation of this approach for an intervention study on heart failure. RESULTS: A total of 1,022 patients were screened, with 223 (23%) of patients being found eligible for enrollment into the intervention study. The iterative approach improved the phenotyping in each screening cycle. Without an iterative approach, the positive screening rate (PSR) was expected to dip below the 20% measured in the first cycle; however, the cyclical approach increased the PSR to 23%. CONCLUSIONS: Our study demonstrates that dynamic phenotyping can facilitate recruitment for prospective clinical study. Future directions include improved informatics infrastructure and governance policies to enable real-time updates to research repositories, tooling for EHR annotation, and methodologies to reduce human annotation.

Towards optimal electronic medical record use: perspectives of advanced users.

Background: While primary health care electronic medical record (EMR) adoption has increased in Canada, the use of advanced EMR features is limited. Realizing the potential benefits of primary health care EMR use is dependent not only on EMR acquisition, but also on its comprehensive use and integration into practice; yet, little is known about the advanced use of EMRs in primary health care. Objective: To explore the views of advanced primary health care EMR users practising in a team-based environment. Methods: A descriptive qualitative approach was used to explore the views of primary health care practitioners who were identified as advanced EMR users. Twelve individual semi-structured interviews were held with primary health care practitioners in Southwestern Ontario, Canada. Field notes were created after each interview. Interviews were audio recorded and transcribed verbatim. Researchers independently coded the transcripts and then met to discuss the results of the coding. We used a thematic approach to data analysis. Results: Three themes emerged from the data analysis: advanced EMR users as individuals with signature characteristics, advanced EMR users as visionaries and advanced EMR users as agents of change. In any one participant, these elements could overlap, illuminating the important interplay between these themes. Taken together, these themes defined advanced use among this group of primary health care practitioners. Conclusions: To realize the potential benefits of EMR use in improved patient care and outcomes, we need to understand how to support EMR use. This study provides a necessary building block in furthering this understanding.

Reliability of ultrasound shear-wave elastography in assessing low back musculature elasticity in asymptomatic individuals.

Patients with low back pain commonly exhibit impaired morphology and function of spinal musculature that may be quantifiable using shear-wave elastography (SWE). The purpose of this study was to assess the intra-rater and test-retest reliability of SWE elasticity measures of the lumbar erector spinae and multifidus muscles during rest and differing levels of contraction in asymptomatic individuals. This single-group repeated-measures design involved a baseline measurement session and a follow-up session 3 days later. The lumbar multifidus was imaged at rest and during three levels of contraction (minimal, moderate, and maximum). The lumbar erector spinae (illiocostalis and longissimus muscles) were imaged at rest only. Overall reliability estimates were fair to excellent with ICCs ranging from 0.44 to 0.92. Reliability was higher in the lumbar multifidus muscles than the erector spinae muscles, slightly higher during contraction than during rest, and substantially improved by using the mean of 3 measurements. By reliably quantifying impaired spinal musculature, SWE may facilitate an improved understanding of the etiology and treatment of low back pain and other muscle pain-related conditions such as trigger points and fibromyalgia.

Information Technology Support for Clinical Genetic Testing within an Academic Medical Center.

Academic medical centers require many interconnected systems to fully support genetic testing processes. We provide an overview of the end-to-end support that has been established surrounding a genetic testing laboratory within our environment, including both laboratory and clinician facing infrastructure. We explain key functions that we have found useful in the supporting systems. We also consider ways that this infrastructure could be enhanced to enable deeper assessment of genetic test results in both the laboratory and clinic.

Stratigraphy of the Anthropocene.

The Anthropocene, an informal term used to signal the impact of collective human activity on biological, physical and chemical processes on the Earth system, is assessed using stratigraphic criteria. It is complex in time, space and process, and may be considered in terms of the scale, relative timing, duration and novelty of its various phenomena. The lithostratigraphic signal includes both direct components, such as urban constructions and man-made deposits, and indirect ones, such as sediment flux changes. Already widespread, these are producing a significant 'event layer', locally with considerable long-term preservation potential. Chemostratigraphic signals include new organic compounds, but are likely to be dominated by the effects of CO(2) release, particularly via acidification in the marine realm, and man-made radionuclides. The sequence stratigraphic signal is negligible to date, but may become geologically significant over centennial/millennial time scales. The rapidly growing biostratigraphic signal includes geologically novel aspects (the scale of globally transferred species) and geologically will have permanent effects.