ABSTRACT
Since authorization of the Moderna mRNA COVID-19 Vaccine, real-world evidence has indicated its effectiveness in preventing COVID-19 cases. However, increased cases of mRNA vaccine-associated myocarditis/pericarditis have been reported, predominantly in young adults and adolescents. The Food and Drug Administration conducted benefit-risk assessment to inform review of the Biologics License Application for use of the Moderna vaccine among individuals ages 18 years and older. We modeled benefit-risk per million individuals who receive two complete doses of the vaccine. Benefit endpoints were vaccine-preventable COVID-19 cases, hospitalizations, intensive care unit (ICU) admissions, and deaths. The risk endpoints were vaccine-related myocarditis/pericarditis cases, hospitalizations, ICU admissions and deaths. The analysis was conducted on the age-stratified male population, due to data signals and previous work showing males to be the main risk group. We constructed six scenarios to evaluate the impact of uncertainty associated with pandemic dynamics, vaccine effectiveness (VE) against novel variants, and rates of vaccine-associated myocarditis/pericarditis cases on the model results. For our most likely scenario, we assumed the US COVID-19 incidence was for the week of December 25, 2021, and a VE of 30% against cases and 72% against hospitalization with the Omicron-dominant strain. Our source for estimating vaccine-attributable myocarditis/pericarditis rates was FDAs CBER Biologics Effectiveness and Safety (BEST) System databases. Overall, our results supported the conclusion that the benefits of the vaccine outweigh its risks. Remarkably, we predicted vaccinating one million 18-25 year-old males would prevent 82,484 cases, 4,766 hospitalizations, 1,144 ICU admissions, and 51 deaths due to COVID-19, comparing to 128 vaccine-attributable myocarditis/pericarditis cases, 110 hospitalizations, zero ICU admissions, and zero deaths. Uncertainties in the pandemic trajectory, effectiveness of vaccine against novel variants, and vaccine-attributable myocarditis/pericarditis rate are important limitations of our analysis. Also, the model does not evaluate potential long-term adverse effects due to either COVID-19 or vaccine-attributable myocarditis/pericarditis.
ABSTRACT
ImportanceActive monitoring of health outcomes following COVID-19 vaccination offers early detection of rare outcomes that may not be identified in pre-licensure trials. ObjectiveTo conduct near-real time monitoring of health outcomes following BNT162b2 COVID-19 vaccination in the U.S. pediatric population aged 5-17 years. DesignWe conducted rapid cycle analysis of 20 pre-specified health outcomes, 13 of which underwent sequential testing and 7 of which were monitored descriptively within a cohort of vaccinated individuals. We tested for increased risk of each health outcome following vaccination compared to a historical baseline, while adjusting for repeated looks at the data as well as claims processing delay. SettingThis is a population-based study in three large commercial claims databases conducted under the U.S. FDA public health surveillance mandate. ParticipantsThe study included over 3 million enrollees aged 5-17 years with BNT162b2 COVID-19 vaccination through mid-2022 in three commercial claims databases. We required continuous enrollment in a medical health insurance plan from the start of an outcome-specific clean window to the COVID-19 vaccination. ExposureExposure was defined as receipt of a BNT162b2 COVID-19 vaccine dose. The primary analysis assessed primary series doses together (Dose 1 + Dose 2), and dose-specific secondary analyses were conducted. Follow up time was censored for death, disenrollment, end of risk window, end of study period, or a subsequent vaccine dose. Main Outcome(s) and Measure(s)We monitored 20 pre-specified health outcomes. We performed descriptive monitoring for all outcomes and sequential testing for 13 outcomes. ResultsAmong 13 health outcomes evaluated by sequential testing, 12 did not meet the threshold for a statistical signal in any of the three databases. In our primary analysis, myocarditis/pericarditis signaled following primary series vaccination with BNT162b2 in ages 12-17 years across all three databases. Conclusions and RelevanceConsistent with published literature, our near-real time monitoring identified a signal for only myocarditis/pericarditis following BNT162b2 COVID-19 vaccination in children aged 12-17 years. This method is intended for early detection of safety signals. Our results are reassuring of the safety of the vaccine, and the potential benefits of vaccination outweigh the risks. Key PointsO_ST_ABSQuestionC_ST_ABSDid active monitoring detect potentially elevated risk of health outcomes following BNT162b2 COVID-19 vaccination in the U.S. pediatric population aged 5-17 years? FindingsTwelve of 13 health outcomes did not meet the safety signal threshold following BNT162b2 COVID-19 vaccination in three large commercial claims databases using near real-time monitoring. Myocarditis/pericarditis met the statistical threshold for a signal following primary series vaccination in ages 12-17 years. MeaningResults from near-real time monitoring of health outcomes following BNT162b2 COVID-19 vaccination provide additional reassuring evidence of vaccine safety in the pediatric population. The myocarditis/pericarditis signal is consistent with current evidence and is being further evaluated.
ABSTRACT
Quantifying how accurate epidemiological models of COVID-19 forecast the number of future cases and deaths can help frame how to incorporate mathematical models to inform public health decisions. Here we analyze and score the predictive ability of publicly available COVID-19 epidemiological models on the COVID-19 Forecast Hub. Our score uses the posted forecast cumulative distributions to compute the log-likelihood for held-out COVID-19 positive cases and deaths. Scores are updated continuously as new data become available, and model performance is tracked over time. We use model scores to construct ensemble models based on past performance. Our publicly available quantitative framework may aid in improving modeling frameworks, and assist policy makers in selecting modeling paradigms to balance the delicate trade-offs between the economy and public health.
