Scenario design for infectious disease projections: Integrating concepts from decision analysis and experimental design.

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, situational awareness, horizon scanning, forecasting, and value of information) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

Who is missed in a community-based survey: Assessment and implications of biases due to incomplete sampling frame in a community-based serosurvey, Choma and Ndola Districts, Zambia, 2022.

Community-based serological studies are increasingly relied upon to measure disease burden, identify population immunity gaps, and guide control and elimination strategies; however, there is little understanding of the potential for and impact of sampling biases on outcomes of interest. As part of efforts to quantify measles immunity gaps in Zambia, a community-based serological survey using stratified multi-stage cluster sampling approach was conducted in Ndola and Choma districts in May-June 2022, enrolling 1245 individuals. We carried out a follow-up study among individuals missed from the sampling frame of the serosurvey in July-August 2022, enrolling 672 individuals. We assessed the potential for and impact of biases in the community-based serosurvey by i) estimating differences in characteristics of households and individuals included and excluded (77% vs 23% of households) from the sampling frame of the serosurvey and ii) evaluating the magnitude these differences make on healthcare-seeking behavior, vaccination coverage, and measles seroprevalence. We found that missed households were 20% smaller and 25% less likely to have children. Missed individuals resided in less wealthy households, had different distributions of sex and occupation, and were more likely to seek care at health facilities. Despite these differences, simulating a survey in which missed households were included in the sampling frame resulted in less than a 5% estimated bias in these outcomes. Although community-based studies are upheld as the gold standard study design in assessing immunity gaps and underlying community health characteristics, these findings underscore the fact that sampling biases can impact the results of even well-conducted community-based surveys. Results from these studies should be interpreted in the context of the study methodology and challenges faced during implementation, which include shortcomings in establishing accurate and up-to-date sampling frames. Failure to account for these shortcomings may result in biased estimates and detrimental effects on decision-making.

flepiMoP: The evolution of a flexible infectious disease modeling pipeline during the COVID-19 pandemic.

The COVID-19 pandemic led to an unprecedented demand for projections of disease burden and healthcare utilization under scenarios ranging from unmitigated spread to strict social distancing policies. In response, members of the Johns Hopkins Infectious Disease Dynamics Group developed flepiMoP (formerly called the COVID Scenario Modeling Pipeline), a comprehensive open-source software pipeline designed for creating and simulating compartmental models of infectious disease transmission and inferring parameters through these models. The framework has been used extensively to produce short-term forecasts and longer-term scenario projections of COVID-19 at the state and county level in the US, for COVID-19 in other countries at various geographic scales, and more recently for seasonal influenza. In this paper, we highlight how the flepiMoP has evolved throughout the COVID-19 pandemic to address changing epidemiological dynamics, new interventions, and shifts in policy-relevant model outputs. As the framework has reached a mature state, we provide a detailed overview of flepiMoP's key features and remaining limitations, thereby distributing flepiMoP and its documentation as a flexible and powerful tool for researchers and public health professionals to rapidly build and deploy large-scale complex infectious disease models for any pathogen and demographic setup.

Ensemble2: Scenarios ensembling for communication and performance analysis.

Throughout the COVID-19 pandemic, scenario modeling played a crucial role in shaping the decision-making process of public health policies. Unlike forecasts, scenario projections rely on specific assumptions about the future that consider different plausible states-of-the-world that may or may not be realized and that depend on policy interventions, unpredictable changes in the epidemic outlook, etc. As a consequence, long-term scenario projections require different evaluation criteria than the ones used for traditional short-term epidemic forecasts. Here, we propose a novel ensemble procedure for assessing pandemic scenario projections using the results of the Scenario Modeling Hub (SMH) for COVID-19 in the United States (US). By defining a "scenario ensemble" for each model and the ensemble of models, termed "Ensemble2", we provide a synthesis of potential epidemic outcomes, which we use to assess projections' performance, bypassing the identification of the most plausible scenario. We find that overall the Ensemble2 models are well-calibrated and provide better performance than the scenario ensemble of individual models. The ensemble procedure accounts for the full range of plausible outcomes and highlights the importance of scenario design and effective communication. The scenario ensembling approach can be extended to any scenario design strategy, with potential refinements including weighting scenarios and allowing the ensembling process to evolve over time.

The US COVID-19 and Influenza Scenario Modeling Hubs: Delivering long-term projections to guide policy.

Between December 2020 and April 2023, the COVID-19 Scenario Modeling Hub (SMH) generated operational multi-month projections of COVID-19 burden in the US to guide pandemic planning and decision-making in the context of high uncertainty. This effort was born out of an attempt to coordinate, synthesize and effectively use the unprecedented amount of predictive modeling that emerged throughout the COVID-19 pandemic. Here we describe the history of this massive collective research effort, the process of convening and maintaining an open modeling hub active over multiple years, and attempt to provide a blueprint for future efforts. We detail the process of generating 17 rounds of scenarios and projections at different stages of the COVID-19 pandemic, and disseminating results to the public health community and lay public. We also highlight how SMH was expanded to generate influenza projections during the 2022-23 season. We identify key impacts of SMH results on public health and draw lessons to improve future collaborative modeling efforts, research on scenario projections, and the interface between models and policy.

Scenario Design for Infectious Disease Projections: Integrating Concepts from Decision Analysis and Experimental Design.

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, value of information, situational awareness, horizon scanning, and forecasting) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

Changes in mobility patterns during the COVID-19 pandemic in Zambia: Implications for the effectiveness of NPIs in Sub-Saharan Africa.

The COVID-19 pandemic has impacted many facets of human behavior, including human mobility partially driven by the implementation of non-pharmaceutical interventions (NPIs) such as stay at home orders, travel restrictions, and workplace and school closures. Given the importance of human mobility in the transmission of SARS-CoV-2, there have been an increase in analyses of mobility data to understand the COVID-19 pandemic to date. However, despite an abundance of these analyses, few have focused on Sub-Saharan Africa (SSA). Here, we use mobile phone calling data to provide a spatially refined analysis of sub-national human mobility patterns during the COVID-19 pandemic from March 2020-July 2021 in Zambia using transmission and mobility models. Overall, among highly trafficked intra-province routes, mobility decreased up to 52% during the time of the strictest NPIs (March-May 2020) compared to baseline. However, despite dips in mobility during the first wave of COVID-19 cases, mobility returned to baseline levels and did not drop again suggesting COVID-19 cases did not influence mobility in subsequent waves.

Context-dependent representation of within- and between-model uncertainty: aggregating probabilistic predictions in infectious disease epidemiology.

Probabilistic predictions support public health planning and decision making, especially in infectious disease emergencies. Aggregating outputs from multiple models yields more robust predictions of outcomes and associated uncertainty. While the selection of an aggregation method can be guided by retrospective performance evaluations, this is not always possible. For example, if predictions are conditional on assumptions about how the future will unfold (e.g. possible interventions), these assumptions may never materialize, precluding any direct comparison between predictions and observations. Here, we summarize literature on aggregating probabilistic predictions, illustrate various methods for infectious disease predictions via simulation, and present a strategy for choosing an aggregation method when empirical validation cannot be used. We focus on the linear opinion pool (LOP) and Vincent average, common methods that make different assumptions about between-prediction uncertainty. We contend that assumptions of the aggregation method should align with a hypothesis about how uncertainty is expressed within and between predictions from different sources. The LOP assumes that between-prediction uncertainty is meaningful and should be retained, while the Vincent average assumes that between-prediction uncertainty is akin to sampling error and should not be preserved. We provide an R package for implementation. Given the rising importance of multi-model infectious disease hubs, our work provides useful guidance on aggregation and a deeper understanding of the benefits and risks of different approaches.

Comparison of allocation strategies of convalescent plasma to reduce excess infections and mortality from SARS-CoV-2 in a US-like population.

BACKGROUND: While the use of convalescent plasma (CP) in the ongoing COVID-19 pandemic has been inconsistent, CP has the potential to reduce excess morbidity and mortality in future pandemics. Given constraints on CP supply, decisions surrounding the allocation of CP must be made. STUDY DESIGN AND METHODS: Using a discrete-time stochastic compartmental model, we simulated implementation of four potential allocation strategies: administering CP to individuals in early hospitalization with COVID-19; administering CP to individuals in outpatient settings; administering CP to hospitalized individuals and administering any remaining CP to outpatient individuals and administering CP in both settings while prioritizing outpatient individuals. We examined the final size of SARS-CoV-2 infections, peak and cumulative hospitalizations, and cumulative deaths under each of the allocation scenarios over a 180-day period. We compared the cost per weighted health benefit under each strategy. RESULTS: Prioritizing administration to patients in early hospitalization, with remaining plasma administered in outpatient settings, resulted in the highest reduction in mortality, averting on average 15% more COVID-19 deaths than administering to hospitalized individuals alone (95% CI [11%-18%]). Prioritizing administration to outpatients, with remaining plasma administered to hospitalized individuals, had the highest percentage of hospitalizations averted (22% [21%-23%] higher than administering to hospitalized individuals alone). DISCUSSION: Convalescent plasma allocation strategy should be determined by the relative priority of averting deaths, infections, or hospitalizations. Under conditions considered, mixed allocation strategies (allocating CP to both outpatient and hospitalized individuals) resulted in a larger percentage of infections and deaths averted than administering CP in a single setting.

Evaluation of FluSight influenza forecasting in the 2021-22 and 2022-23 seasons with a new target laboratory-confirmed influenza hospitalizations.

Accurate forecasts can enable more effective public health responses during seasonal influenza epidemics. Forecasting teams were asked to provide national and jurisdiction-specific probabilistic predictions of weekly confirmed influenza hospital admissions for one through four weeks ahead for the 2021-22 and 2022-23 influenza seasons. Across both seasons, 26 teams submitted forecasts, with the submitting teams varying between seasons. Forecast skill was evaluated using the Weighted Interval Score (WIS), relative WIS, and coverage. Six out of 23 models outperformed the baseline model across forecast weeks and locations in 2021-22 and 12 out of 18 models in 2022-23. Averaging across all forecast targets, the FluSight ensemble was the 2nd most accurate model measured by WIS in 2021-22 and the 5th most accurate in the 2022-23 season. Forecast skill and 95% coverage for the FluSight ensemble and most component models degraded over longer forecast horizons and during periods of rapid change. Current influenza forecasting efforts help inform situational awareness, but research is needed to address limitations, including decreased performance during periods of changing epidemic dynamics.

Feasibility of measles and rubella vaccination programmes for disease elimination: a modelling study.

BACKGROUND: Marked reductions in the incidence of measles and rubella have been observed since the widespread use of the measles and rubella vaccines. Although no global goal for measles eradication has been established, all six WHO regions have set measles elimination targets. However, a gap remains between current control levels and elimination targets, as shown by large measles outbreaks between 2017 and 2019. We aimed to model the potential for measles and rubella elimination globally to inform a WHO report to the 73rd World Health Assembly on the feasibility of measles and rubella eradication. METHODS: In this study, we modelled the probability of measles and rubella elimination between 2020 and 2100 under different vaccination scenarios in 93 countries of interest. We evaluated measles and rubella burden and elimination across two national transmission models each (Dynamic Measles Immunisation Calculation Engine [DynaMICE], Pennsylvania State University [PSU], Johns Hopkins University, and Public Health England models), and one subnational measles transmission model (Institute for Disease Modeling model). The vaccination scenarios included a so-called business as usual approach, which continues present vaccination coverage, and an intensified investment approach, which increases coverage into the future. The annual numbers of infections projected by each model, country, and vaccination scenario were used to explore if, when, and for how long the infections would be below a threshold for elimination. FINDINGS: The intensified investment scenario led to large reductions in measles and rubella incidence and burden. Rubella elimination is likely to be achievable in all countries and measles elimination is likely in some countries, but not all. The PSU and DynaMICE national measles models estimated that by 2050, the probability of elimination would exceed 75% in 14 (16%) and 36 (39%) of 93 modelled countries, respectively. The subnational model of measles transmission highlighted inequity in routine coverage as a likely driver of the continuance of endemic measles transmission in a subset of countries. INTERPRETATION: To reach regional elimination goals, it will be necessary to innovate vaccination strategies and technologies that increase spatial equity of routine vaccination, in addition to investing in existing surveillance and outbreak response programmes. FUNDING: WHO, Gavi, the Vaccine Alliance, US Centers for Disease Control and Prevention, and the Bill & Melinda Gates Foundation.

An evaluation of prospective COVID-19 modelling studies in the USA: from data to science translation.

Infectious disease modelling can serve as a powerful tool for situational awareness and decision support for policy makers. However, COVID-19 modelling efforts faced many challenges, from poor data quality to changing policy and human behaviour. To extract practical insight from the large body of COVID-19 modelling literature available, we provide a narrative review with a systematic approach that quantitatively assessed prospective, data-driven modelling studies of COVID-19 in the USA. We analysed 136 papers, and focused on the aspects of models that are essential for decision makers. We have documented the forecasting window, methodology, prediction target, datasets used, and geographical resolution for each study. We also found that a large fraction of papers did not evaluate performance (25%), express uncertainty (50%), or state limitations (36%). To remedy some of these identified gaps, we recommend the adoption of the EPIFORGE 2020 model reporting guidelines and creating an information-sharing system that is suitable for fast-paced infectious disease outbreak science.

Projected resurgence of COVID-19 in the United States in July-December 2021 resulting from the increased transmissibility of the Delta variant and faltering vaccination.

In Spring 2021, the highly transmissible SARS-CoV-2 Delta variant began to cause increases in cases, hospitalizations, and deaths in parts of the United States. At the time, with slowed vaccination uptake, this novel variant was expected to increase the risk of pandemic resurgence in the US in summer and fall 2021. As part of the COVID-19 Scenario Modeling Hub, an ensemble of nine mechanistic models produced 6-month scenario projections for July-December 2021 for the United States. These projections estimated substantial resurgences of COVID-19 across the US resulting from the more transmissible Delta variant, projected to occur across most of the US, coinciding with school and business reopening. The scenarios revealed that reaching higher vaccine coverage in July-December 2021 reduced the size and duration of the projected resurgence substantially, with the expected impacts was largely concentrated in a subset of states with lower vaccination coverage. Despite accurate projection of COVID-19 surges occurring and timing, the magnitude was substantially underestimated 2021 by the models compared with the of the reported cases, hospitalizations, and deaths occurring during July-December, highlighting the continued challenges to predict the evolving COVID-19 pandemic. Vaccination uptake remains critical to limiting transmission and disease, particularly in states with lower vaccination coverage. Higher vaccination goals at the onset of the surge of the new variant were estimated to avert over 1.5 million cases and 21,000 deaths, although may have had even greater impacts, considering the underestimated resurgence magnitude from the model.

A Systematic Review of Coronavirus Disease 2019 Vaccine Efficacy and Effectiveness Against Severe Acute Respiratory Syndrome Coronavirus 2 Infection and Disease.

Billions of doses of coronavirus disease 2019 (COVID-19) vaccines have been administered globally, dramatically reducing severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) incidence and severity in some settings. Many studies suggest vaccines provide a high degree of protection against infection and disease, but precise estimates vary and studies differ in design, outcomes measured, dosing regime, location, and circulating virus strains. In this study, we conduct a systematic review of COVID-19 vaccines through February 2022. We included efficacy data from Phase 3 clinical trials for 15 vaccines undergoing World Health Organization Emergency Use Listing evaluation and real-world effectiveness for 8 vaccines with observational studies meeting inclusion criteria. Vaccine metrics collected include protection against asymptomatic infection, any infection, symptomatic COVID-19, and severe outcomes including hospitalization and death, for partial or complete vaccination, and against variants of concern Alpha, Beta, Gamma, Delta, and Omicron. We additionally review the epidemiological principles behind the design and interpretation of vaccine efficacy and effectiveness studies, including important sources of heterogeneity.

Clostridioides difficile infection (CDI) in a previous room occupant predicts CDI in subsequent room occupants across different hospital settings.

BACKGROUND: Previous single-center studies suggest that exposure to a room previously occupied by a patient with CDI may increase the risk of CDI in subsequent patients. We evaluated the risk of previous room occupant on CDI risk across 5 adult hospitals. METHODS: This is a non-concurrent cohort study of adult inpatients admitted to 5 hospitals. Exposed rooms were identified as being occupied by a patient diagnosed with CDI and a logistic regression was performed to assess if staying in an exposed room increases the risk of CDI in subsequent patients. RESULTS: Patients admitted to a room that was previously occupied by a patient with CDI had a 27% increased odds of subsequently being diagnosed with CDI (odds ratio (OR)=1.269; 95% confidence interval (CI)= 1.12-1.44) if exposed within the last 90 days and 40% increased odds (OR=1.401; 95% CI= 1.25-1.57) if exposed in the last 365 days after controlling for previous admissions and length of stay. Cumulative patient-day exposure to previously CDI-positive occupied rooms within both 90 and 365 days were also found to be independently significant, with a 4.5% (OR 1.045; 95% CI = 1.03-1.06) and 4.2% (OR 1.042; 95% CI = 1.03-1.06) increase in odds of CDI with each day of exposure respectively. DISCUSSION/CONCLUSIONS: This study adds further evidence that hospital environment in patient rooms may contribute to risk for CDI.

COVID-19 vaccine hesitancy in Zambia: a glimpse at the possible challenges ahead for COVID-19 vaccination rollout in sub-Saharan Africa.

With unprecedented speed, multiple vaccines against SARS-CoV-2 are available 1 year after the COVID-19 pandemic was first identified. As we push to achieve global control through these new vaccines, old challenges present themselves, including cold-chain storage, the logistics of mass vaccination, and vaccine hesitancy. Understanding how much hesitancy toward COVID-19 vaccines might occur and what factors may be driving these concerns can improve the ability of public health workers and communicators to maximize vaccine uptake. We nested a survey within a measles-rubella mass vaccination campaign in Zambia in November 2020 and asked about sentiments and beliefs toward COVID-19 and COVID-19 vaccines. Among parents bringing their children to receive a measles-rubella vaccine, we found high acceptability of COVID-19 vaccination of their children, but substantial uncertainty and hesitancy about receiving the vaccine themselves. COVID-19 vaccination hesitancy was correlated with beliefs around COVID-19 severity and risk, as well as vaccine safety and effectiveness.

Projected resurgence of COVID-19 in the United States in July-December 2021 resulting from the increased transmissibility of the Delta variant and faltering vaccination.

WHAT IS ALREADY KNOWN ABOUT THIS TOPIC?: The highly transmissible SARS-CoV-2 Delta variant has begun to cause increases in cases, hospitalizations, and deaths in parts of the United States. With slowed vaccination uptake, this novel variant is expected to increase the risk of pandemic resurgence in the US in July-December 2021. WHAT IS ADDED BY THIS REPORT?: Data from nine mechanistic models project substantial resurgences of COVID-19 across the US resulting from the more transmissible Delta variant. These resurgences, which have now been observed in most states, were projected to occur across most of the US, coinciding with school and business reopening. Reaching higher vaccine coverage in July-December 2021 reduces the size and duration of the projected resurgence substantially. The expected impact of the outbreak is largely concentrated in a subset of states with lower vaccination coverage. WHAT ARE THE IMPLICATIONS FOR PUBLIC HEALTH PRACTICE?: Renewed efforts to increase vaccination uptake are critical to limiting transmission and disease, particularly in states with lower current vaccination coverage. Reaching higher vaccination goals in the coming months can potentially avert 1.5 million cases and 21,000 deaths and improve the ability to safely resume social contacts, and educational and business activities. Continued or renewed non-pharmaceutical interventions, including masking, can also help limit transmission, particularly as schools and businesses reopen.

Lives saved with vaccination for 10 pathogens across 112 countries in a pre-COVID-19 world.

Background: Vaccination is one of the most effective public health interventions. We investigate the impact of vaccination activities for Haemophilus influenzae type b, hepatitis B, human papillomavirus, Japanese encephalitis, measles, Neisseria meningitidis serogroup A, rotavirus, rubella, Streptococcus pneumoniae, and yellow fever over the years 2000-2030 across 112 countries. Methods: Twenty-one mathematical models estimated disease burden using standardised demographic and immunisation data. Impact was attributed to the year of vaccination through vaccine-activity-stratified impact ratios. Results: We estimate 97 (95%CrI[80, 120]) million deaths would be averted due to vaccination activities over 2000-2030, with 50 (95%CrI[41, 62]) million deaths averted by activities between 2000 and 2019. For children under-5 born between 2000 and 2030, we estimate 52 (95%CrI[41, 69]) million more deaths would occur over their lifetimes without vaccination against these diseases. Conclusions: This study represents the largest assessment of vaccine impact before COVID-19-related disruptions and provides motivation for sustaining and improving global vaccination coverage in the future. Funding: VIMC is jointly funded by Gavi, the Vaccine Alliance, and the Bill and Melinda Gates Foundation (BMGF) (BMGF grant number: OPP1157270 / INV-009125). Funding from Gavi is channelled via VIMC to the Consortium's modelling groups (VIMC-funded institutions represented in this paper: Imperial College London, London School of Hygiene and Tropical Medicine, Oxford University Clinical Research Unit, Public Health England, Johns Hopkins University, The Pennsylvania State University, Center for Disease Analysis Foundation, Kaiser Permanente Washington, University of Cambridge, University of Notre Dame, Harvard University, Conservatoire National des Arts et Métiers, Emory University, National University of Singapore). Funding from BMGF was used for salaries of the Consortium secretariat (authors represented here: TBH, MJ, XL, SE-L, JT, KW, NMF, KAMG); and channelled via VIMC for travel and subsistence costs of all Consortium members (all authors). We also acknowledge funding from the UK Medical Research Council and Department for International Development, which supported aspects of VIMC's work (MRC grant number: MR/R015600/1).JHH acknowledges funding from National Science Foundation Graduate Research Fellowship; Richard and Peggy Notebaert Premier Fellowship from the University of Notre Dame. BAL acknowledges funding from NIH/NIGMS (grant number R01 GM124280) and NIH/NIAID (grant number R01 AI112970). The Lives Saved Tool (LiST) receives funding support from the Bill and Melinda Gates Foundation.This paper was compiled by all coauthors, including two coauthors from Gavi. Other funders had no role in study design, data collection, data analysis, data interpretation, or writing of the report. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.