Estimating COVID-19 Vaccination Effectiveness Using Electronic Health Records of an Academic Medical Center in Michigan

ABSTRACT

ImportanceSystematic characterization of the protective effect of vaccinations across time and at-risk populations is needed to inform public health guidelines and personalized interventions. ObjectiveTo evaluate the vaccine effectiveness (VE) over time and determine differences across demographic and clinical risk factors of COVID-19. Design, Setting, and ParticipantsThis test negative design consisted of adult patients who were tested or diagnosed for COVID-19 at Michigan Medicine in 2021. Variables extracted from Electronic Health Records included vaccination status, age, gender, race/ethnicity, comorbidities, body mass index, residential-level socioeconomic characteristics, past COVID-19 infection, being immunosuppressed, and health care worker status. ExposureThe primary exposure was vaccination status and was categorized into fully vaccinated with and without booster, partially vaccinated, or unvaccinated. Main Outcomes and MeasuresThe main outcomes were infection with COVID-19 (positive test or diagnosis) and having severe COVID-19, i.e., either being hospitalized or deceased. Based on these, VE was calculated by quarter, vaccine, and patient characteristics. ResultsOf 170,487 COVID-19 positive adult patients, 78,002 (45.8%) were unvaccinated, and 92,485 (54.2%) were vaccinated, among which 74,060 (80.1%) were fully vaccinated. COVID-19 positivity and severity rates were substantially higher among unvaccinated (12.1% and 1.4%, respectively) compared to fully vaccinated individuals (4.7% and 0.4%, respectively). Among 7,187 individuals with a booster, only 18 (0.3%) had a severe outcome. The covariate-adjusted VE against an infection was 62.1% (95%CI 60.3-63.8%), being highest in the Q2 of 2021 (90.9% [89.5-92.1%]), lowest in Q3 (60.1% [55.9-64.0%]), and rebounding in Q4 to 68.8% [66.3- 71.1%]). Similarly, VE against severe COVID-19 overall was 73.7% (69.6-77.3%) and remained high throughout 2021 87.4% (58.1-96.3%), 92.2% (88.3-94.8%), 74.4% (64.8-81.5%) and 83.0% (78.8-86.4%), respectively. Data on fully vaccinated individuals from Q4 indicated additional protection against infection with an additional booster dose (VE-Susceptibility 64.0% [61.1-66.7%] vs. 87.3% [85.0-89.2%]) and severe outcomes (VE-Severity 78.8% [73.5-83.0%] vs. 94.0% [89.5-96.6%]). Comparisons between Pfizer-BioNTech and Moderna vaccines indicated similar protection against susceptibility (82.9% [80.7-84.9%] versus 88.1% [85.5- 90.2%]) and severity (87.1% [80.3-91.6%]) vs. (84.9% [76.2-90.5%]) after controlling for vaccination timing and other factors. There was no significant effect modification by all the factors we examined. Conclusions and RelevanceOur findings suggest that COVID-19 vaccines offered high protection against infection and severe COVID-19, and showed decreasing effectiveness over time and improved protection with a booster. Key PointsO_ST_ABSQuestionC_ST_ABSHow do the rates of COVID-19 outcomes (infections or mild/severe disease) compare across vaccination status and quarters of 2021, after adjusting for confounders? FindingsIn this cohort of 170,487 adult patients tested for or diagnosed with COVID-19 during 2021, both COVID-19 positivity and severity rates were substantially higher in unvaccinated compared to fully vaccinated individuals. Vaccine effectiveness estimation was adjusted for covariates potentially related to both being vaccinated and COVID-19 outcomes; this also allowed us to determine if effectiveness differed across patient subgroups. The estimated vaccine effectiveness across the four quarters of 2021 was 62.1% against infection and was 73.7% against severe COVID-19 (defined as hospitalization, ICU admission, or death). There was no significant effect modification by all the factors we examined. MeaningThese findings suggest COVID-19 vaccines had relatively high protection against infection and severe COVID-19 during 2021 for those who received two doses of an mRNA vaccine (Moderna or Pfizer-BioNTech) or one dose of the Janssen vaccine, of which the effectiveness decreased over time and improved with a booster.