The Impact of Variability in Patient Exposure During Premarket Clinical Development on Postmarket Safety Outcomes.

We characterized the size of the premarket safety population for 278 small-molecule new molecular entities (NMEs) and 61 new therapeutic biologics (NTBs) approved by the US Food and Drug Administration (FDA) between October 1, 2002, and December 31, 2014, evaluating the relationship of premarket safety population size to regulatory characteristics and postmarket safety outcomes. The median size of the safety population was 1,044, and was lower for NTBs than NMEs (median: 920 vs. 1,138, P = 0.04), orphan products than nonorphan products (393 vs. 1,606, P < 0.001), and for products with fast-track designation (617 vs. 1,455, P < 0.001), priority review (630 vs. 1,735, P < 0.001), and accelerated approval (475 vs. 1,164, P < 0.001), than products without that designation. The median number of postmarket safety label updates and issues added to the label were higher with larger premarket exposure among nonorphan products, but not among orphan products. Products with accelerated approval using a surrogate end point had a higher median number of safety issues added to the label than those with full approval, but this did not vary with the size of the safety population; fast-track and priority review were not associated with the number of safety issues added to the label. A smaller safety population size was associated with a longer time to first safety outcome for nonorphan products but not orphan products. For orphan and nonorphan products combined, smaller premarket safety population size is not associated with the number or timing of postmarket safety outcomes, regardless of expedited program participation.

ADE Eval: An Evaluation of Text Processing Systems for Adverse Event Extraction from Drug Labels for Pharmacovigilance.

INTRODUCTION: The US FDA is interested in a tool that would enable pharmacovigilance safety evaluators to automate the identification of adverse drug events (ADEs) mentioned in FDA prescribing information. The MITRE Corporation (MITRE) and the FDA organized a shared task-Adverse Drug Event Evaluation (ADE Eval)-to determine whether the performance of algorithms currently used for natural language processing (NLP) might be good enough for real-world use. OBJECTIVE: ADE Eval was conducted to evaluate a range of NLP techniques for identifying ADEs mentioned in publicly available FDA-approved drug labels (package inserts). It was designed specifically to reflect pharmacovigilance practices within the FDA and model possible pharmacovigilance use cases. METHODS: Pharmacovigilance-specific annotation guidelines and annotated corpora were created. Two metrics modeled the experiences of FDA safety evaluators: one measured the ability of an algorithm to identify correct Medical Dictionary for Regulatory Activities (MedDRA®) terms for the text from the annotated corpora, and the other assessed the quality of evidence extracted from the corpora to support the selected MedDRA® term by measuring the portion of annotated text an algorithm correctly identified. A third metric assessed the cost of correcting system output for subsequent training (averaged, weighted F1-measure for mention finding). RESULTS: In total, 13 teams submitted 23 runs: the top MedDRA® coding F1-measure was 0.79, the top quality score was 0.96, and the top mention-finding F1-measure was 0.89. CONCLUSION: While NLP techniques do not perform at levels that would allow them to be used without intervention, it is now worthwhile exploring making NLP outputs available in human pharmacovigilance workflows.

Postmarketing Safety-Related Regulatory Actions for New Therapeutic Biologics Approved in the United States 2002-2014: Similarities and Differences With New Molecular Entities.

We examined the relationship of regulatory and review characteristics to postmarketing safety-related regulatory actions for 61 new therapeutic biologics (NTBs) approved between October 1, 2002 and December 31, 2014. We also compared NTBs with small-molecule new molecular entities (NMEs) on these measures. Postmarketing safety-related regulatory actions were defined as a safety-related withdrawal or a safety-related update to a safety section of the label through June 30, 2018. Four NTBs were withdrawn, two for safety reasons. At least one safety-related update was added to the labels of 54 (88.5%) NTBs. Label updates occurred throughout the follow-up period. Time to the first safety-related regulatory action was shorter for NTBs approved under accelerated approval. The occurrence of safety events was more likely to occur with NTBs than with NMEs. This may be explained in part by the higher proportion of NTBs in the anatomical therapeutic chemical classification categories with higher frequency of safety-related updates. NTBs also had shorter time to safety events than NMEs. These findings underscore the importance of continued development of the life cycle safety surveillance system for both drugs and biologics with consideration for product type and its characteristics, including pharmacologic action.

Validating a Framework for Coding Patient-Reported Health Information to the Medical Dictionary for Regulatory Activities Terminology: An Evaluative Study.

BACKGROUND: The availability of and interest in patient-generated health data (PGHD) have grown steadily. Patients describe medical experiences differently compared with how clinicians or researchers would describe their observations of those same experiences. Patients may find nonserious, known adverse drug events (ADEs) to be an ongoing concern, which impacts the tolerability and adherence. Clinicians must be vigilant for medically serious, potentially fatal ADEs. Having both perspectives provides patients and clinicians with a complete picture of what to expect from drug therapies. Multiple initiatives seek to incorporate patients' perspectives into drug development, including PGHD exploration for pharmacovigilance. The Food and Drug Administration (FDA) Adverse Event Reporting System contains case reports of postmarketing ADEs. To facilitate the analysis of these case reports, case details are coded using the Medical Dictionary for Regulatory Activities (MedDRA). PatientsLikeMe is a Web-based network where patients report, track, share, and discuss their health information. PatientsLikeMe captures PGHD through free-text and structured data fields. PatientsLikeMe structured data are coded to multiple medical terminologies, including MedDRA. The standardization of PatientsLikeMe PGHD enables electronic accessibility and enhances patient engagement. OBJECTIVE: The aim of this study is to retrospectively review PGHD for symptoms and ADEs entered by patients on PatientsLikeMe and coded by PatientsLikeMe to MedDRA terminology for concordance with regulatory-focused coding practices. METHODS: An FDA MedDRA coding expert retrospectively reviewed a data file containing verbatim patient-reported symptoms and ADEs and PatientsLikeMe-assigned MedDRA terms to determine the medical accuracy and appropriateness of the selected MedDRA terms, applying the International Council for Harmonisation MedDRA Term Selection: Points to Consider (MTS:PTC) guides. RESULTS: The FDA MedDRA coding expert reviewed 3234 PatientsLikeMe-assigned MedDRA codes and patient-reported verbatim text. The FDA and PatientsLikeMe were concordant at 97.09% (3140/3234) of the PatientsLikeMe-assigned MedDRA codes. The 2.91% (94/3234) discordant subset was analyzed to identify reasons for differences. Coding differences were attributed to several reasons but mostly driven by PatientsLikeMe's approach of assigning a more general MedDRA term to enable patient-to-patient engagement, while the FDA assigned a more specific medically relevant term. CONCLUSIONS: PatientsLikeMe MedDRA coding of PGHD was generally comparable to how the FDA would code similar data, applying the MTS:PTC principles. Discordant coding resulted from several reasons but mostly reflected a difference in purpose. The MTS:PTC coding principles aim to capture the most specific reported information about an ADE, whereas PatientsLikeMe may code patient-reported symptoms and ADEs to more general MedDRA terms to support patient engagement among a larger group of patients. This study demonstrates that most verbatim reports of symptoms and ADEs collected by a PGHD source, such as the PatientsLikeMe platform, could be reliably coded to MedDRA terminology by applying the MTS:PTC guide. Regarding all secondary use of novel data, understanding coding and standardization principles applied to these data types are important.

Evaluation of Natural Language Processing (NLP) systems to annotate drug product labeling with MedDRA terminology.

INTRODUCTION: The FDA Adverse Event Reporting System (FAERS) is a primary data source for identifying unlabeled adverse events (AEs) in a drug or biologic drug product's postmarketing phase. Many AE reports must be reviewed by drug safety experts to identify unlabeled AEs, even if the reported AEs are previously identified, labeled AEs. Integrating the labeling status of drug product AEs into FAERS could increase report triage and review efficiency. Medical Dictionary for Regulatory Activities (MedDRA) is the standard for coding AE terms in FAERS cases. However, drug manufacturers are not required to use MedDRA to describe AEs in product labels. We hypothesized that natural language processing (NLP) tools could assist in automating the extraction and MedDRA mapping of AE terms in drug product labels. MATERIALS AND METHODS: We evaluated the performance of three NLP systems, (ETHER, I2E, MetaMap) for their ability to extract AE terms from drug labels and translate the terms to MedDRA Preferred Terms (PTs). Pharmacovigilance-based annotation guidelines for extracting AE terms from drug labels were developed for this study. We compared each system's output to MedDRA PT AE lists, manually mapped by FDA pharmacovigilance experts using the guidelines, for ten drug product labels known as the "gold standard AE list" (GSL) dataset. Strict time and configuration conditions were imposed in order to test each system's capabilities under conditions of no human intervention and minimal system configuration. Each NLP system's output was evaluated for precision, recall and F measure in comparison to the GSL. A qualitative error analysis (QEA) was conducted to categorize a random sample of each NLP system's false positive and false negative errors. RESULTS: A total of 417, 278, and 250 false positive errors occurred in the ETHER, I2E, and MetaMap outputs, respectively. A total of 100, 80, and 187 false negative errors occurred in ETHER, I2E, and MetaMap outputs, respectively. Precision ranged from 64% to 77%, recall from 64% to 83% and F measure from 67% to 79%. I2E had the highest precision (77%), recall (83%) and F measure (79%). ETHER had the lowest precision (64%). MetaMap had the lowest recall (64%). The QEA found that the most prevalent false positive errors were context errors such as "Context error/General term", "Context error/Instructions or monitoring parameters", "Context error/Medical history preexisting condition underlying condition risk factor or contraindication", and "Context error/AE manifestations or secondary complication". The most prevalent false negative errors were in the "Incomplete or missed extraction" error category. Missing AE terms were typically due to long terms, or terms containing non-contiguous words which do not correspond exactly to MedDRA synonyms. MedDRA mapping errors were a minority of errors for ETHER and I2E but were the most prevalent false positive errors for MetaMap. CONCLUSIONS: The results demonstrate that it may be feasible to use NLP tools to extract and map AE terms to MedDRA PTs. However, the NLP tools we tested would need to be modified or reconfigured to lower the error rates to support their use in a regulatory setting. Tools specific for extracting AE terms from drug labels and mapping the terms to MedDRA PTs may need to be developed to support pharmacovigilance. Conducting research using additional NLP systems on a larger, diverse GSL would also be informative.

Postmarket Safety Outcomes for New Molecular Entity (NME) Drugs Approved by the Food and Drug Administration Between 2002 and 2014.

We ascertained a comprehensive list of postmarket safety outcomes, defined as a safety-related market withdrawal or an update to a safety-related section of product label for 278 new molecular entity drugs (NMEs) with a follow-up period of up to 13 years. At least one safety-related update was added to 195 (70.1%) labels of the drugs studied. Updates occurred as early as 160 days after approval and throughout the follow-up period. The period between the second and eighth postapproval year was the most active, with a slight attenuation thereafter. The times to the first safety outcome were significantly shorter for NMEs approved with a fast-track designation (P = 0.02) or under an accelerated approval using a surrogate endpoint (P = 0.03). Our findings underscore the importance of a robust safety surveillance system throughout a drug's lifecycle and for practitioners and patients to remain updated on drug safety profiles.

Quality assessment of spontaneous triggered adverse event reports received by the Food and Drug Administration.

PURPOSE: The Food and Drug Administration (FDA) conducted a quality assessment of the Adverse Drug Events Spontaneous Triggered Event Reporting (ASTER) pilot study, which represented the FDA's first experience with the receipt of electronic health record (EHR)-triggered adverse event reports. The EHR-triggered adverse event reports from ASTER were evaluated for their utility in conducting FDA's pharmacovigilance work. FDA is sharing these findings to assist others who are pursuing the use of patient EHR data for electronic adverse event identification and reporting. METHODS: ASTER pilot study reports were identified from the FDA Adverse Event Reporting System database, then reviewed and assessed. RESULTS: Demographic and other objective data that can be easily derived from EHRs were both present in the submitted reports and relevant to the reported adverse drug event (ADE), but other data, such as an informative description of the ADE, dates that support a temporal relationship between the product and the event, and relevant laboratory data, were often either conflicting or lacking. Most of the ADEs captured in the ASTER pilot and reported to FDA are known events (i.e. included in product labeling) for the suspect drugs. CONCLUSION: Triggered adverse event reporting from patient EHRs is a potentially valuable source of postmarketing safety information, especially for known adverse events. Attention to quality is needed to ensure that the data generated from EHR-triggered ADE reporting systems are relevant to the reported adverse events so that the FDA and others engaged in pharmacovigilance can fully utilize these reports.