Monitoring Drug Safety in Pregnancy with Scan Statistics: A Comparison of Two Study Designs.

BACKGROUND: Traditional surveillance of adverse infant outcomes following maternal medication exposures relies on pregnancy exposure registries, which are often underpowered. We characterize the statistical power of TreeScan, a data mining tool, to identify potential signals in the setting of perinatal medication exposures and infant outcomes. METHODS: We used empirical data to inform background incidence of major congenital malformations and other birth conditions. Statistical power was calculated using two probability models compatible with TreeScan, Bernoulli and Poisson, while varying the sample size, magnitude of the risk increase, and incidence of a specified outcome. We also simulated larger referent to exposure matching ratios when using the Bernoulli model in the setting of fixed N:1 propensity score matching. Finally, we assessed the impact of outcome misclassification on power. RESULTS: The Poisson model demonstrated greater power to detect signals than the Bernoulli model across all scenarios and suggested a sample size of 4,000 exposed pregnancies is needed to detect a twofold increase in risk of a common outcome (approximately 8 per 1,000) with 85% power. Increasing the fixed matching ratio with the Bernoulli model did not reliably increase power. An outcome definition with high sensitivity is expected to have somewhat greater power to detect signals than an outcome definition with high positive predictive value. CONCLUSIONS: Use of the Poisson model with an outcome definition that prioritizes sensitivity may be optimal for signal detection. TreeScan is a viable method for surveillance of adverse infant outcomes following maternal medication use.

Novel methods for pregnancy drug safety surveillance in the FDA Sentinel System.

PURPOSE: It is a priority of the US Food and Drug Administration (FDA) to monitor the safety of medications used during pregnancy. Pregnancy exposure registries and cohort studies utilizing electronic health record data are primary sources of information but are limited by small sample sizes and limited outcome assessment. TreeScan™, a statistical data mining tool, can be applied within the FDA Sentinel System to simultaneously identify multiple potential adverse neonatal and infant outcomes after maternal medication exposure. METHODS: We implemented TreeScan using the Sentinel analytic tools in a cohort of linked live birth deliveries and infants nested in the IBM MarketScan® Research Database. As a case study, we compared first trimester fluoroquinolone use and cephalosporin use. We used the Bernoulli and Poisson TreeScan statistics with compatible propensity score-based study designs for confounding control (matching and stratification) and used multiple propensity score models with various strategies for confounding control to inform best practices. We developed a hierarchical outcome tree including major congenital malformations and outcomes of gestational length and birth weight. RESULTS: A total of 1791 fluoroquinolone-exposed and 8739 cephalosporin-exposed mother-infant pairs were eligible for analysis. Both TreeScan analysis methods resulted in single alerts that were deemed to be due to uncontrolled confounding or otherwise not warranting follow-up. CONCLUSIONS: In this implementation of TreeScan using Sentinel analytic tools, we did not observe any new safety signals for fluoroquinolone use in the first trimester. TreeScan, with tailored or high-dimensional propensity scores for confounding control, is a valuable tool in addition to current safety surveillance methods for medications used during pregnancy.

Validation of an ICD-10-based algorithm to identify stillbirth in the Sentinel System.

PURPOSE: To develop and validate an International Classification of Diseases, 10th Revision, Clinical Modification (ICD-10-CM)-based algorithm to identify cases of stillbirth using electronic healthcare data. METHODS: We conducted a retrospective study using claims data from three Data Partners (healthcare systems and insurers) in the Sentinel Distributed Database. Algorithms were developed using ICD-10-CM diagnosis codes to identify potential stillbirths among females aged 12-55 years between July 2016 and June 2018. A random sample of medical charts (N = 169) was identified for chart abstraction and adjudication. Two physician adjudicators reviewed potential cases to determine whether a stillbirth event was definite/probable, the date of the event, and the gestational age at delivery. Positive predictive values (PPVs) were calculated for the algorithms. Among confirmed cases, agreement between the claims data and medical charts was determined for the outcome date and gestational age at stillbirth. RESULTS: Of the 110 potential cases identified, adjudicators determined that 54 were stillbirth events. Criteria for the algorithm with the highest PPV (82.5%; 95% CI, 70.9%-91.0%) included the presence of a diagnosis code indicating gestational age ≥20 weeks and occurrence of either >1 stillbirth-related code or no other pregnancy outcome code (i.e., livebirth, spontaneous abortion, induced abortion) recorded on the index date. We found ≥90% agreement within 7 days between the claims data and medical charts for both the outcome date and gestational age at stillbirth. CONCLUSIONS: Our results suggest that electronic healthcare data may be useful for signal detection of medical product exposures potentially associated with stillbirth.

Validation of an electronic algorithm for Hodgkin and non-Hodgkin lymphoma in ICD-10-CM.

PURPOSE: Lymphoma is a health outcome of interest for drug safety studies. Studies using administrative claims data require the accurate identification of lymphoma cases. We developed and validated an International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM)-based algorithm to identify lymphoma in healthcare claims data. METHODS: We developed a three-component algorithm to identify patients aged ≥15 years who were newly diagnosed with Hodgkin (HL) or non-Hodgkin (NHL) lymphoma from January 2016 through July 2018 among members of four Data Partners within the FDA's Sentinel System. The algorithm identified potential cases as patients with ≥2 ICD-10-CM lymphoma diagnosis codes on different dates within 183 days; ≥1 procedure code for a diagnostic procedure (e.g., biopsy, flow cytometry) and ≥1 procedure code for a relevant imaging study within 90 days of the first lymphoma diagnosis code. Cases identified by the algorithm were adjudicated via chart review and a positive predictive value (PPV) was calculated. RESULTS: We identified 8723 potential lymphoma cases via the algorithm and randomly sampled 213 for validation. We retrieved 138 charts (65%) and adjudicated 134 (63%). The overall PPV was 77% (95% confidence interval: 69%-84%). Most cases also had subtype information available, with 88% of cases identified as NHL and 11% as HL. CONCLUSIONS: Seventy-seven percent of lymphoma cases identified by an algorithm based on ICD-10-CM diagnosis and procedure codes and applied to claims data were true cases. This novel algorithm represents an efficient, cost-effective way to target an important health outcome of interest for large-scale drug safety and public health surveillance studies.

A General Propensity Score for Signal Identification Using Tree-Based Scan Statistics.

The tree-based scan statistic (TreeScan; Martin Kulldorff, Harvard Medical School, Boston, Massachusetts) is a data-mining method that adjusts for multiple testing of correlated hypotheses when screening thousands of potential adverse events for signal identification. Simulation has demonstrated the promise of TreeScan with a propensity score (PS)-matched cohort design. However, it is unclear which variables to include in a PS for applied signal identification studies to simultaneously adjust for confounding across potential outcomes. We selected 4 pairs of medications with well-understood safety profiles. For each pair, we evaluated 5 candidate PSs with different combinations of 1) predefined general covariates (comorbidity, frailty, utilization), 2) empirically selected (data-driven) covariates, and 3) covariates tailored to the drug pair. For each pair, statistical alerting patterns were similar with alternative PSs (≤11 alerts in 7,996 outcomes scanned). Inclusion of covariates tailored to exposure did not appreciably affect screening results. Inclusion of empirically selected covariates can provide better proxy coverage for confounders but can also decrease statistical power. Unlike tailored covariates, empirical and predefined general covariates can be applied "out of the box" for signal identification. The choice of PS depends on the level of concern about residual confounding versus loss of power. Potential signals should be followed by pharmacoepidemiologic assessment where confounding control is tailored to the specific outcome(s) under investigation.

The risk of febrile seizures following influenza and 13-valent pneumococcal conjugate vaccines.

BACKGROUND: Evidence on the risk of febrile seizures after inactivated influenza vaccine (IIV) and 13-valent pneumococcal conjugate vaccine (PCV13) is mixed. In the FDA-sponsored Sentinel Initiative, we examined risk of febrile seizures after IIV and PCV13 in children 6-23 months of age during the 2013-14 and 2014-15 influenza seasons. METHODS: Using claims data and a self-controlled risk interval design, we compared the febrile seizure rate in a risk interval (0-1 days) versus control interval (14-20 days). In exploratory analyses, we assessed whether the effect of IIV was modified by concomitant PCV13 administration. RESULTS: Adjusted for age, calendar time and concomitant administration of the other vaccine, the incidence rate ratio (IRR) for risk of febrile seizures following IIV was 1.12 (95% CI 0.80, 1.56) and following PCV13 was 1.80 (95% CI 1.29, 2.52). The attributable risk for febrile seizures following PCV13 ranged from 0.33 to 5.16 per 100,000 doses by week of age. The age and calendar-time adjusted IRR comparing exposed to unexposed time was numerically larger for concomitant IIV and PCV13 (IRR 2.80, 95% CI 1.63, 4.83), as compared to PCV13 without concomitant IIV (IRR 1.54, 95% CI 1.04, 2.28), and the IRR for IIV without concomitant PCV13 suggested no independent effects of IIV (IRR 0.94, 95% CI 0.63, 1.42). Taken together, this suggests a possible interaction between IIV and PCV13, though our study was not sufficiently powered to provide a precise estimate of the interaction. CONCLUSIONS: We found an elevated risk of febrile seizures after PCV13 vaccine but not after IIV. The risk of febrile seizures after PCV13 is low compared to the overall risk in this population of children, and the risk should be interpreted in the context of the importance of preventing pneumococcal infections.

Kawasaki disease and 13-valent pneumococcal conjugate vaccination among young children: A self-controlled risk interval and cohort study with null results.

BACKGROUND: Kawasaki disease is an acute vasculitis that primarily affects children younger than 5 years of age. Its etiology is unknown. The United States Vaccine Safety Datalink conducted postlicensure safety surveillance for 13-valent pneumococcal conjugate vaccine (PCV13), comparing the risk of Kawasaki disease within 28 days of PCV13 vaccination with the historical risk after 7-valent PCV (PCV7) vaccination and using chart-validation. A relative risk (RR) of 2.38 (95% CI 0.92-6.38) was found. Concurrently, the Food and Drug Administration (FDA) conducted a postlicensure safety review that identified cases of Kawasaki disease through adverse event reporting. The FDA decided to initiate a larger study of Kawasaki disease risk following PCV13 vaccination in the claims-based Sentinel/Postlicensure Rapid Immunization Safety Monitoring (PRISM) surveillance system. The objective of this study was to determine the existence and magnitude of any increased risk of Kawasaki disease in the 28 days following PCV13 vaccination. METHODS AND FINDINGS: The study population included mostly commercially insured children from birth to <24 months of age in 2010 to 2015 from across the US. Using claims data of participating Sentinel/PRISM data-providing organizations, PCV13 vaccinations were identified by means of current procedural terminology (CPT), Healthcare Common Procedure Coding System (HCPCS), and National Drug Code (NDC) codes. Potential cases of Kawasaki disease were identified by first-in-365-days International Classification of Diseases 9th revision (ICD-9) code 446.1 or International Classification of Diseases 10th revision (ICD-10) code M30.3 in the inpatient setting. Medical records were sought for potential cases and adjudicated by board-certified pediatricians. The primary analysis used chart-confirmed cases with adjudicated symptom onset in a self-controlled risk interval (SCRI) design, which controls for time-invariant potential confounders. The prespecified risk interval was Days 1-28 after vaccination; a 28-day-long control interval followed this risk interval. A secondary analytic approach used a cohort design, with alternative potential risk intervals of Days 1-28 and Days 1-42. The varying background risk of Kawasaki disease by age was adjusted for in both designs. In the primary analysis, there were 43 confirmed cases of Kawasaki disease in the risk interval and 44 in the control interval. The age-adjusted risk estimate was 1.07 (95% CI 0.70-1.63; p = 0.76). In the secondary, cohort analyses, which included roughly 700 potential cases and more than 3 million person-years, the risk estimates of potential Kawasaki disease in the risk interval versus in unexposed person-time were 0.84 (95% CI 0.65-1.08; p = 0.18) for the Days 1-28 risk interval and 0.97 (95% CI 0.79-1.19; p = 0.80) for the Days 1-42 risk interval. The main limitation of the study was that we lacked the resources to conduct medical record review for all the potential cases of Kawasaki disease. As a result, potential cases rather than chart-confirmed cases were used in the cohort analyses. CONCLUSIONS: With more than 6 million doses of PCV13 administered, no evidence was found of an association between PCV13 vaccination and Kawasaki disease onset in the 4 weeks after vaccination nor of an elevated risk extending or concentrated somewhat beyond 4 weeks. These null results were consistent across alternative designs, age-adjustment methods, control intervals, and categories of Kawasaki disease case included.