Development of Interoperable Computable Phenotype Algorithms for Adverse Events of Special Interest to Be Used for Biologics Safety Surveillance: Validation Study.

BACKGROUND: Adverse events associated with vaccination have been evaluated by epidemiological studies and more recently have gained additional attention with the emergency use authorization of several COVID-19 vaccines. As part of its responsibility to conduct postmarket surveillance, the US Food and Drug Administration continues to monitor several adverse events of special interest (AESIs) to ensure vaccine safety, including for COVID-19. OBJECTIVE: This study is part of the Biologics Effectiveness and Safety Initiative, which aims to improve the Food and Drug Administration's postmarket surveillance capabilities while minimizing public burden. This study aimed to enhance active surveillance efforts through a rules-based, computable phenotype algorithm to identify 5 AESIs being monitored by the Center for Disease Control and Prevention for COVID-19 or other vaccines: anaphylaxis, Guillain-Barré syndrome, myocarditis/pericarditis, thrombosis with thrombocytopenia syndrome, and febrile seizure. This study examined whether these phenotypes have sufficiently high positive predictive value (PPV) to ensure that the cases selected for surveillance are reasonably likely to be a postbiologic adverse event. This allows patient privacy, and security concerns for the data sharing of patients who had nonadverse events can be properly accounted for when evaluating the cost-benefit aspect of our approach. METHODS: AESI phenotype algorithms were developed to apply to electronic health record data at health provider organizations across the country by querying for standard and interoperable codes. The codes queried in the rules represent symptoms, diagnoses, or treatments of the AESI sourced from published case definitions and input from clinicians. To validate the performance of the algorithms, we applied them to electronic health record data from a US academic health system and provided a sample of cases for clinicians to evaluate. Performance was assessed using PPV. RESULTS: With a PPV of 93.3%, our anaphylaxis algorithm performed the best. The PPVs for our febrile seizure, myocarditis/pericarditis, thrombocytopenia syndrome, and Guillain-Barré syndrome algorithms were 89%, 83.5%, 70.2%, and 47.2%, respectively. CONCLUSIONS: Given our algorithm design and performance, our results support continued research into using interoperable algorithms for widespread AESI postmarket detection.

Data quality and timeliness analysis for post-vaccination adverse event cases reported through healthcare data exchange to FDA BEST pilot platform.

Introduction: This study is part of the U.S. Food and Drug Administration (FDA)'s Biologics Effectiveness and Safety (BEST) initiative, which aims to improve the FDA's postmarket surveillance capabilities by using real-world data (RWD). In the United States, using RWD for postmarket surveillance has been hindered by the inability to exchange clinical data between healthcare providers and public health organizations in an interoperable format. However, the Office of the National Coordinator for Health Information Technology (ONC) has recently enacted regulation requiring all healthcare providers to support seamless access, exchange, and use of electronic health information through the interoperable HL7 Fast Healthcare Interoperability Resources (FHIR) standard. To leverage the recent ONC changes, BEST designed a pilot platform to query and receive the clinical information necessary to analyze suspected AEs. This study assessed the feasibility of using the RWD received through the data exchange of FHIR resources to study post-vaccination AE cases by evaluating the data volume, query response time, and data quality. Materials and methods: The study used RWD from 283 post-vaccination AE cases, which were received through the platform. We used descriptive statistics to report results and apply 322 data quality tests based on a data quality framework for EHR. Results: The volume analysis indicated the average clinical resources for a post-vaccination AE case was 983.9 for the median partner. The query response time analysis indicated that cases could be received by the platform at a median of 3 min and 30 s. The quality analysis indicated that most of the data elements and conformance requirements useful for postmarket surveillance were met. Discussion: This study describes the platform's data volume, data query response time, and data quality results from the queried postvaccination adverse event cases and identified updates to current standards to close data quality gaps.

A Versatile and Scalable Platform That Streamlines Data Collection for Patient-Centered Studies: Usability and Feasibility Study.

BACKGROUND: The Food and Drug Administration Center for Biologics Evaluation and Research (CBER) established the Biologics Effectiveness and Safety (BEST) Initiative with several objectives, including the expansion and enhancement of CBER's access to fit-for-purpose data sources, analytics, tools, and infrastructures to improve the understanding of patient experiences with conditions related to CBER-regulated products. Owing to existing challenges in data collection, especially for rare disease research, CBER recognized the need for a comprehensive platform where study coordinators can engage with study participants and design and deploy studies while patients or caregivers could enroll, consent, and securely participate as well. OBJECTIVE: This study aimed to increase awareness and describe the design, development, and novelty of the Survey of Health and Patient Experience (SHAPE) platform, its functionality and application, quality improvement efforts, open-source availability, and plans for enhancement. METHODS: SHAPE is hosted in a Google Cloud environment and comprises 3 parts: the administrator application, participant app, and application programming interface. The administrator can build a study comprising a set of questionnaires and self-report entries through the app. Once the study is deployed, the participant can access the app, consent to the study, and complete its components. To build SHAPE to be scalable and flexible, we leveraged the open-source software development kit, Ionic Framework. This enabled the building and deploying of apps across platforms, including iOS, Android, and progressive web applications, from a single codebase by using standardized web technologies. SHAPE has been integrated with a leading Health Level 7 (HL7®) Fast Healthcare Interoperability Resources (FHIR®) application programming interface platform, 1upHealth, which allows participants to consent to 1-time data pull of their electronic health records. We used an agile-based process that engaged multiple stakeholders in SHAPE's design and development. RESULTS: SHAPE allows study coordinators to plan, develop, and deploy questionnaires to obtain important end points directly from patients or caregivers. Electronic health record integration enables access to patient health records, which can validate and enhance the accuracy of data-capture methods. The administrator can then download the study data into HL7® FHIR®-formatted JSON files. In this paper, we illustrate how study coordinators can use SHAPE to design patient-centered studies. We demonstrate its broad applicability through a hypothetical type 1 diabetes cohort study and an ongoing pilot study on metachromatic leukodystrophy to implement best practices for designing a regulatory-grade natural history study for rare diseases. CONCLUSIONS: SHAPE is an intuitive and comprehensive data-collection tool for a variety of clinical studies. Further customization of this versatile and scalable platform allows for multiple use cases. SHAPE can capture patient perspectives and clinical data, thereby providing regulators, clinicians, researchers, and patient advocacy organizations with data to inform drug development and improve patient outcomes.

Detection of allergic transfusion-related adverse events from electronic medical records.

BACKGROUND: Transfusion-related adverse events can be unrecognized and unreported. As part of the US Food and Drug Administration's Center for Biologics Evaluation and Research Biologics Effectiveness and Safety initiative, we explored whether machine learning methods, such as natural language processing (NLP), can identify and report transfusion allergic reactions (ARs) from electronic health records (EHRs). STUDY DESIGN AND METHODS: In a 4-year period, all 146 reported transfusion ARs were pulled from a database of 86,764 transfusions in an academic health system, along with a random sample of 605 transfusions without reported ARs. Structured and unstructured EHR data were retrieved, including demographics, new symptoms, medications, and lab results. In unstructured data, evidence from clinicians' notes, test results, and prescriptions fields identified transfusion ARs, which were used to extract NLP features. Clinician reviews of selected validation cases assessed and confirmed model performance. RESULTS: Clinician reviews of selected validation cases yielded a sensitivity of 67.9% and a specificity of 97.5% at a threshold of 0.9, with a positive predictive value (PPV) of 84%, estimated to 4.5% when extrapolated to match transfusion AR incidence in the full transfusion dataset. A higher threshold achieved sensitivity of 43% with specificity/PPV of 100% in our validation set. Essential features predicting ARs were recognized transfusion reactions, administration of antihistamines or glucocorticoids, and skin symptoms (e.g., hives and itching). Removal of NLP features decreased model performance. DISCUSSION: NLP algorithms can identify transfusion reactions from the EHR with a reasonable level of precision for subsequent clinician review and confirmation.

The Food and Drug Administration Biologics Effectiveness and Safety Initiative Facilitates Detection of Vaccine Administrations From Unstructured Data in Medical Records Through Natural Language Processing.

Introduction: The Food and Drug Administration Center for Biologics Evaluation and Research conducts post-market surveillance of biologic products to ensure their safety and effectiveness. Studies have found that common vaccine exposures may be missing from structured data elements of electronic health records (EHRs), instead being captured in clinical notes. This impacts monitoring of adverse events following immunizations (AEFIs). For example, COVID-19 vaccines have been regularly administered outside of traditional medical settings. We developed a natural language processing (NLP) algorithm to mine unstructured clinical notes for vaccinations not captured in structured EHR data. Methods: A random sample of 1,000 influenza vaccine administrations, representing 995 unique patients, was extracted from a large U.S. EHR database. NLP techniques were used to detect administrations from the clinical notes in the training dataset [80% (N = 797) of patients]. The algorithm was applied to the validation dataset [20% (N = 198) of patients] to assess performance. Full medical charts for 28 randomly selected administration events in the validation dataset were reviewed by clinicians. The NLP algorithm was then applied across the entire dataset (N = 995) to quantify the number of additional events identified. Results: A total of 3,199 administrations were identified in the structured data and clinical notes combined. Of these, 2,740 (85.7%) were identified in the structured data, while the NLP algorithm identified 1,183 (37.0%) administrations in clinical notes; 459 were not also captured in the structured data. This represents a 16.8% increase in the identification of vaccine administrations compared to using structured data alone. The validation of 28 vaccine administrations confirmed 27 (96.4%) as "definite" vaccine administrations; 18 (64.3%) had evidence of a vaccination event in the structured data, while 10 (35.7%) were found solely in the unstructured notes. Discussion: We demonstrated the utility of an NLP algorithm to identify vaccine administrations not captured in structured EHR data. NLP techniques have the potential to improve detection of vaccine administrations not otherwise reported without increasing the analysis burden on physicians or practitioners. Future applications could include refining estimates of vaccine coverage and detecting other exposures, population characteristics, and outcomes not reliably captured in structured EHR data.

The Effects of Birth Year, Age and Sex on Hemagglutination Inhibition Antibody Responses to Influenza Vaccination.

The first exposure to influenza is thought to impact subsequent immune responses later in life. The consequences of this can be seen during influenza epidemics and pandemics with differences in morbidity and mortality for different birth cohorts. There is a need for better understanding of how vaccine responses are affected by early exposures to influenza viruses. In this analysis of hemagglutination inhibition (HI) antibody responses in two cohorts of military personnel we noticed differences related to age, sex, prior vaccination, deployment and birth year. These data suggest that HI antibody production, in response to influenza vaccination, is affected by these factors. The magnitude of this antibody response is associated with, among other factors, the influenza strain that circulated following birth.

An Inter-regional US Blood Supply Simulation Model to Evaluate Blood Availability to Support Planning for Emergency Preparedness and Medical Countermeasures.

OBJECTIVES: Planning for a response to threats like pandemics or mass casualty events is a national priority. The US blood supply system can be particularly vulnerable to such events. It is important to understand the impacts of emergency situations on blood availability and the resiliency of the US blood supply system. METHODS: On the basis of the Stock-and-Flow simulation model of the US blood supply system, we developed an inter-regional blood transfer system representing the action of multiple blood collectors and distributors to enable effective planning of strategies to minimize collection and donation disruptions to the blood supply system in the event of a national emergency. RESULTS: We simulated a pandemic or mass casualty event on both a national and an inter-regional blood supply system. Differences in the estimated impacts demonstrated the importance of incorporating spatial and temporal variations of blood collection and utilization across US regions. The absence of blood shortage in both emergency scenarios highlighted the resilience of the inter-regional system to meet the potential associated blood demand. CONCLUSIONS: Our inter-regional model considered complex factors and can be a valuable tool to assist regulatory decision-making and strategic planning for emergency preparedness to avoid and mitigate associated adverse health consequences. (Disaster Med Public Health Preparedness. 2018;12:201-210).

Modeling the potential impact on the US blood supply of transfusing critically ill patients with fresher stored red blood cells.

BACKGROUND: Although some studies have suggested that transfusion recipients may have better medical outcomes if transfused with red blood cell units stored for a short time, the overall body of evidence shows mixed results. It is important to understand how using fresher stored red blood cell units for certain patient groups may affect blood availability. METHODS: Based on the Stock-and-Flow simulation model of the US blood supply developed by Simonetti et al. 2014, we evaluated a newly implemented allocation method of preferentially transfusing fresher stored red blood cell units to a subset of high-risk group of critically ill patients and its potential impact on supply. RESULTS: Simulation results showed that, depending on the scenario, the US blood total supply might be reduced between 2-42%, when compared to the standard of care in transfusion medicine practice. Among our simulated scenarios, we observed that the number of expired red blood cell units modulated the supply levels. The age threshold of the required red blood cell units was inversely correlated with both the supply levels and the number of transfused units that failed to meet that age threshold. CONCLUSION: To our knowledge, this study represents the first attempt to develop a comprehensive framework to evaluate the impact of preferentially transfusing fresher stored red blood cells to the higher-risk critically ill patients on supply. Model results show the difficulties to identify an optimal scenario.

A strain-based model for mechanical hemolysis based on a coarse-grained red blood cell model.

Mechanical hemolysis is a major concern in the design of cardiovascular devices, such as prosthetic heart valves and ventricular assist devices. The primary cause of mechanical hemolysis is the impact of the device on the local blood flow, which exposes blood elements to non-physiologic conditions. The majority of existing hemolysis models correlate red blood cell (RBC) damage to the imposed fluid shear stress and exposure time. Only recently more realistic, strain-based models have been proposed, where the RBC's response to the imposed hydrodynamic loading is accounted for. In the present work we extend strain-based models by introducing a high-fidelity representation of RBCs, which is based on existing coarse-grained particle dynamics approach. We report a series of numerical experiments in simple shear flows of increasing complexity, to illuminate the basic differences between existing models and establish their accuracy in comparison to the high-fidelity RBC approach. We also consider a practical configuration, where the flow through an artificial heart valve is computed. Our results shed light on the strengths and weaknesses of each approach and identify the key gaps that should be addressed in the development of new models.

Modeling of the major gas vesicle protein, GvpA: from protein sequence to vesicle wall structure.

The structure and assembly process of gas vesicles have received significant attention in recent decades, although relatively little is still known. This work combines state-of-the-art computational methods to develop a model for the major gas vesicle protein, GvpA, and explore its structure within the assembled vesicle. Elucidating this protein's structure has been challenging due to its adherent and aggregative nature, which has so far precluded in-depth biochemical analyses. Moreover, GvpA has extremely low similarity with any known protein structure, which renders homology modeling methods ineffective. Thus, alternate approaches were used to model its tertiary structure. Starting with the sequence from haloarchaeon Halobacterium sp. NRC-1, we performed ab initio modeling and threading to acquire a multitude of structure decoys, which were equilibrated and ranked using molecular dynamics and mechanics, respectively. The highest ranked predictions exhibited an α-ß-ß-α secondary structure in agreement with earlier experimental findings, as well as with our own secondary structure predictions. Afterwards, GvpA subunits were docked in a quasi-periodic arrangement to investigate the assembly of the vesicle wall and to conduct simulations of contact-mode atomic force microscopy imaging, which allowed us to reconcile the structure predictions with the available experimental data. Finally, the GvpA structure for two representative organisms, Anabaena flos-aquae and Calothrix sp. PCC 7601, was also predicted, which reproduced the major features of our GvpA model, supporting the expectation that homologous GvpA sequences synthesized by different organisms should exhibit similar structures.