Scenario design for infectious disease projections: Integrating concepts from decision analysis and experimental design.

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, situational awareness, horizon scanning, forecasting, and value of information) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

When do we need multiple infectious disease models? Agreement between projection rank and magnitude in a multi-model setting.

Mathematical models are useful for public health planning and response to infectious disease threats. However, different models can provide differing results, which can hamper decision making if not synthesized appropriately. To address this challenge, multi-model hubs convene independent modeling groups to generate ensembles, known to provide more accurate predictions of future outcomes. Yet, these hubs are resource intensive, and how many models are sufficient in a hub is not known. Here, we compare the benefit of predictions from multiple models in different contexts: (1) decision settings that depend on predictions of quantitative outcomes (e.g., hospital capacity planning), where assessments of the benefits of multi-model ensembles have largely focused; and (2) decisions settings that require the ranking of alternative epidemic scenarios (e.g., comparing outcomes under multiple possible interventions and biological uncertainties). We develop a mathematical framework to mimic a multi-model prediction setting, and use this framework to quantify how frequently predictions from different models agree. We further explore multi-model agreement using real-world, empirical data from 14 rounds of U.S. COVID-19 Scenario Modeling Hub projections. Our results suggest that the value of multiple models could be different in different decision contexts, and if only a few models are available, focusing on the rank of alternative epidemic scenarios could be more robust than focusing on quantitative outcomes. Although additional exploration of the sufficient number of models for different contexts is still needed, our results indicate that it may be possible to identify decision contexts where it is robust to rely on fewer models, a finding that can inform the use of modeling resources during future public health crises.

Scenario Design for Infectious Disease Projections: Integrating Concepts from Decision Analysis and Experimental Design.

Across many fields, scenario modeling has become an important tool for exploring long-term projections and how they might depend on potential interventions and critical uncertainties, with relevance to both decision makers and scientists. In the past decade, and especially during the COVID-19 pandemic, the field of epidemiology has seen substantial growth in the use of scenario projections. Multiple scenarios are often projected at the same time, allowing important comparisons that can guide the choice of intervention, the prioritization of research topics, or public communication. The design of the scenarios is central to their ability to inform important questions. In this paper, we draw on the fields of decision analysis and statistical design of experiments to propose a framework for scenario design in epidemiology, with relevance also to other fields. We identify six different fundamental purposes for scenario designs (decision making, sensitivity analysis, value of information, situational awareness, horizon scanning, and forecasting) and discuss how those purposes guide the structure of scenarios. We discuss other aspects of the content and process of scenario design, broadly for all settings and specifically for multi-model ensemble projections. As an illustrative case study, we examine the first 17 rounds of scenarios from the U.S. COVID-19 Scenario Modeling Hub, then reflect on future advancements that could improve the design of scenarios in epidemiological settings.

Participatory Mathematical Modeling Approach for Policymaking during the First Year of the COVID-19 Crisis, Jordan.

We engaged in a participatory modeling approach with health sector stakeholders in Jordan to support government decision-making regarding implementing public health measures to mitigate COVID-19 disease burden. We considered the effect of 4 physical distancing strategies on reducing COVID-19 transmission and mortality in Jordan during March 2020-January 2021: no physical distancing; intermittent physical distancing where all but essential services are closed once a week; intermittent physical distancing where all but essential services are closed twice a week; and a permanent physical distancing intervention. Modeling showed that the fourth strategy would be most effective in reducing cases and deaths; however, this approach was only marginally beneficial to reducing COVID-19 disease compared with an intermittently enforced physical distancing intervention. Scenario-based model influenced policy-making and the evolution of the pandemic in Jordan confirmed the forecasting provided by the modeling exercise and helped confirm the effectiveness of the policy adopted by the government of Jordan.

Multiple models for outbreak decision support in the face of uncertainty.

Policymakers must make management decisions despite incomplete knowledge and conflicting model projections. Little guidance exists for the rapid, representative, and unbiased collection of policy-relevant scientific input from independent modeling teams. Integrating approaches from decision analysis, expert judgment, and model aggregation, we convened multiple modeling teams to evaluate COVID-19 reopening strategies for a mid-sized United States county early in the pandemic. Projections from seventeen distinct models were inconsistent in magnitude but highly consistent in ranking interventions. The 6-mo-ahead aggregate projections were well in line with observed outbreaks in mid-sized US counties. The aggregate results showed that up to half the population could be infected with full workplace reopening, while workplace restrictions reduced median cumulative infections by 82%. Rankings of interventions were consistent across public health objectives, but there was a strong trade-off between public health outcomes and duration of workplace closures, and no win-win intermediate reopening strategies were identified. Between-model variation was high; the aggregate results thus provide valuable risk quantification for decision making. This approach can be applied to the evaluation of management interventions in any setting where models are used to inform decision making. This case study demonstrated the utility of our approach and was one of several multimodel efforts that laid the groundwork for the COVID-19 Scenario Modeling Hub, which has provided multiple rounds of real-time scenario projections for situational awareness and decision making to the Centers for Disease Control and Prevention since December 2020.

Context-dependent representation of within- and between-model uncertainty: aggregating probabilistic predictions in infectious disease epidemiology.

Probabilistic predictions support public health planning and decision making, especially in infectious disease emergencies. Aggregating outputs from multiple models yields more robust predictions of outcomes and associated uncertainty. While the selection of an aggregation method can be guided by retrospective performance evaluations, this is not always possible. For example, if predictions are conditional on assumptions about how the future will unfold (e.g. possible interventions), these assumptions may never materialize, precluding any direct comparison between predictions and observations. Here, we summarize literature on aggregating probabilistic predictions, illustrate various methods for infectious disease predictions via simulation, and present a strategy for choosing an aggregation method when empirical validation cannot be used. We focus on the linear opinion pool (LOP) and Vincent average, common methods that make different assumptions about between-prediction uncertainty. We contend that assumptions of the aggregation method should align with a hypothesis about how uncertainty is expressed within and between predictions from different sources. The LOP assumes that between-prediction uncertainty is meaningful and should be retained, while the Vincent average assumes that between-prediction uncertainty is akin to sampling error and should not be preserved. We provide an R package for implementation. Given the rising importance of multi-model infectious disease hubs, our work provides useful guidance on aggregation and a deeper understanding of the benefits and risks of different approaches.

Reimagining Cancer: Moving from the Cellular to the Tissue Level.

The current universally accepted explanation of cancer origin and behavior, the somatic mutation theory, is cell-centered and rooted in perturbation of gene function independent of the external environmental context. However, tumors consist of various epithelial and stromal cell populations temporally and spatially organized into an integrated neoplastic community, and they can have properties similar to normal tissues. Accordingly, we review specific normal cellular and tissue traits and behaviors with adaptive temporal and spatial self-organization that result in ordered patterns and structures. A few recent theories have described these tissue-level cancer behaviors, invoking a conceptual shift from the cellular level and highlighting the need for methodologic approaches based on the analysis of complex systems. We propose extending the analytical approach of regulatory networks to the tissue level and introduce the concept of "cancer attractors." These concepts require reevaluation of cancer imaging and investigational approaches and challenge the traditional reductionist approach of cancer molecular biology.

Projected outcomes of universal testing and treatment in a generalised HIV epidemic in Zambia and South Africa (the HPTN 071 [PopART] trial): a modelling study.

BACKGROUND: The long-term impact of universal home-based testing and treatment as part of universal testing and treatment (UTT) on HIV incidence is unknown. We made projections using a detailed individual-based model of the effect of the intervention delivered in the HPTN 071 (PopART) cluster-randomised trial. METHODS: In this modelling study, we fitted an individual-based model to the HIV epidemic and HIV care cascade in 21 high prevalence communities in Zambia and South Africa that were part of the PopART cluster-randomised trial (intervention period Nov 1, 2013, to Dec 31, 2017). The model represents coverage of home-based testing and counselling by age and sex, delivered as part of the trial, antiretroviral therapy (ART) uptake, and any changes in national guidelines on ART eligibility. In PopART, communities were randomly assigned to one of three arms: arm A received the full PopART intervention for all individuals who tested positive for HIV, arm B received the intervention with ART provided in accordance with national guidelines, and arm C received standard of care. We fitted the model to trial data twice using Approximate Bayesian Computation, once before data unblinding and then again after data unblinding. We compared projections of intervention impact with observed effects, and for four different scenarios of UTT up to Jan 1, 2030 in the study communities. FINDINGS: Compared with standard of care, a 51% (95% credible interval 40-60) reduction in HIV incidence is projected if the trial intervention (arms A and B combined) is continued from 2020 to 2030, over and above a declining trend in HIV incidence under standard of care. INTERPRETATION: A widespread and continued commitment to UTT via home-based testing and counselling can have a substantial effect on HIV incidence in high prevalence communities. FUNDING: National Institute of Allergy and Infectious Diseases, US President's Emergency Plan for AIDS Relief, International Initiative for Impact Evaluation, Bill & Melinda Gates Foundation, National Institute on Drug Abuse, and National Institute of Mental Health.

Vote-processing rules for combining control recommendations from multiple models.

Mathematical modelling is used during disease outbreaks to compare control interventions. Using multiple models, the best method to combine model recommendations is unclear. Existing methods weight model projections, then rank control interventions using the combined projections, presuming model outputs are directly comparable. However, the way each model represents the epidemiological system will vary. We apply electoral vote-processing rules to combine model-generated rankings of interventions. Combining rankings of interventions, instead of combining model projections, avoids assuming that projections are comparable as all comparisons of projections are made within each model. We investigate four rules: First-past-the-post, Alternative Vote (AV), Coombs Method and Borda Count. We investigate rule sensitivity by including models that favour only one action or including those that rank interventions randomly. We investigate two case studies: the 2014 Ebola outbreak in West Africa (37 compartmental models) and a hypothetical foot-and-mouth disease outbreak in UK (four individual-based models). The Coombs Method was least susceptible to adding models that favoured a single action, Borda Count and AV were most susceptible to adding models that ranked interventions randomly. Each rule chose the same intervention as when ranking interventions by mean projections, suggesting that combining rankings provides similar recommendations with fewer assumptions about model comparability. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

Estimating SARS-CoV-2 variant fitness and the impact of interventions in England using statistical and geo-spatial agent-based models.

The SARS-CoV-2 epidemic has been extended by the evolution of more transmissible viral variants. In autumn 2020, the B.1.177 lineage became the dominant variant in England, before being replaced by the B.1.1.7 (Alpha) lineage in late 2020, with the sweep occurring at different times in each region. This period coincided with a large number of non-pharmaceutical interventions (e.g. lockdowns) to control the epidemic, making it difficult to estimate the relative transmissibility of variants. In this paper, we model the spatial spread of these variants in England using a meta-population agent-based model which correctly characterizes the regional variation in cases and distribution of variants. As a test of robustness, we additionally estimated the relative transmissibility of multiple variants using a statistical model based on the renewal equation, which simultaneously estimates the effective reproduction number R. Relative to earlier variants, the transmissibility of B.1.177 is estimated to have increased by 1.14 (1.12-1.16) and that of Alpha by 1.71 (1.65-1.77). The vaccination programme starting in December 2020 is also modelled. Counterfactual simulations demonstrate that the vaccination programme was essential for reopening in March 2021, and that if the January lockdown had started one month earlier, up to 30 k (24 k-38 k) deaths could have been prevented. This article is part of the theme issue 'Technical challenges of modelling real-life epidemics and examples of overcoming these'.

Extreme g-Tensor Anisotropy and Its Insensitivity to Structural Distortions in a Family of Linear Two-Coordinate Ni(I) Bis-N-heterocyclic Carbene Complexes.

We report a new series of homoleptic Ni(I) bis-N-heterocyclic carbene complexes with a range of torsion angles between the two ligands from 68° to 90°. Electron paramagnetic resonance measurements revealed a strongly anisotropic g-tensor in all complexes with a small variation in g∥ ∼ 5.7-5.9 and g⊥ ∼ 0.6. The energy of the first excited state identified by variable-field far-infrared magnetic spectroscopy and SOC-CASSCF/NEVPT2 calculations is in the range 270-650 cm-1. Magnetic relaxation measured by alternating current susceptibility up to 10 K is dominated by Raman and direct processes. Ab initio ligand-field analysis reveals that a torsion angle of <90° causes the splitting between doubly occupied dxz and dyz orbitals, which has little effect on the magnetic properties, while the temperature dependence of the magnetic relaxation appears to have no correlation with the torsion angle.

Synergistic interventions to control COVID-19: Mass testing and isolation mitigates reliance on distancing.

Stay-at-home orders and shutdowns of non-essential businesses are powerful, but socially costly, tools to control the pandemic spread of SARS-CoV-2. Mass testing strategies, which rely on widely administered frequent and rapid diagnostics to identify and isolate infected individuals, could be a potentially less disruptive management strategy, particularly where vaccine access is limited. In this paper, we assess the extent to which mass testing and isolation strategies can reduce reliance on socially costly non-pharmaceutical interventions, such as distancing and shutdowns. We develop a multi-compartmental model of SARS-CoV-2 transmission incorporating both preventative non-pharmaceutical interventions (NPIs) and testing and isolation to evaluate their combined effect on public health outcomes. Our model is designed to be a policy-guiding tool that captures important realities of the testing system, including constraints on test administration and non-random testing allocation. We show how strategic changes in the characteristics of the testing system, including test administration, test delays, and test sensitivity, can reduce reliance on preventative NPIs without compromising public health outcomes in the future. The lowest NPI levels are possible only when many tests are administered and test delays are short, given limited immunity in the population. Reducing reliance on NPIs is highly dependent on the ability of a testing program to identify and isolate unreported, asymptomatic infections. Changes in NPIs, including the intensity of lockdowns and stay at home orders, should be coordinated with increases in testing to ensure epidemic control; otherwise small additional lifting of these NPIs can lead to dramatic increases in infections, hospitalizations and deaths. Importantly, our results can be used to guide ramp-up of testing capacity in outbreak settings, allow for the flexible design of combined interventions based on social context, and inform future cost-benefit analyses to identify efficient pandemic management strategies.

PopART-IBM, a highly efficient stochastic individual-based simulation model of generalised HIV epidemics developed in the context of the HPTN 071 (PopART) trial.

Mathematical models are powerful tools in HIV epidemiology, producing quantitative projections of key indicators such as HIV incidence and prevalence. In order to improve the accuracy of predictions, such models need to incorporate a number of behavioural and biological heterogeneities, especially those related to the sexual network within which HIV transmission occurs. An individual-based model, which explicitly models sexual partnerships, is thus often the most natural type of model to choose. In this paper we present PopART-IBM, a computationally efficient individual-based model capable of simulating 50 years of an HIV epidemic in a large, high-prevalence community in under a minute. We show how the model calibrates within a Bayesian inference framework to detailed age- and sex-stratified data from multiple sources on HIV prevalence, awareness of HIV status, ART status, and viral suppression for an HPTN 071 (PopART) study community in Zambia, and present future projections of HIV prevalence and incidence for this community in the absence of trial intervention.

OpenABM-Covid19-An agent-based model for non-pharmaceutical interventions against COVID-19 including contact tracing.

SARS-CoV-2 has spread across the world, causing high mortality and unprecedented restrictions on social and economic activity. Policymakers are assessing how best to navigate through the ongoing epidemic, with computational models being used to predict the spread of infection and assess the impact of public health measures. Here, we present OpenABM-Covid19: an agent-based simulation of the epidemic including detailed age-stratification and realistic social networks. By default the model is parameterised to UK demographics and calibrated to the UK epidemic, however, it can easily be re-parameterised for other countries. OpenABM-Covid19 can evaluate non-pharmaceutical interventions, including both manual and digital contact tracing, and vaccination programmes. It can simulate a population of 1 million people in seconds per day, allowing parameter sweeps and formal statistical model-based inference. The code is open-source and has been developed by teams both inside and outside academia, with an emphasis on formal testing, documentation, modularity and transparency. A key feature of OpenABM-Covid19 are its Python and R interfaces, which has allowed scientists and policymakers to simulate dynamic packages of interventions and help compare options to suppress the COVID-19 epidemic.

[Ni(NHC)2 ] as a Scaffold for Structurally Characterized trans [H-Ni-PR2 ] and trans [R2 P-Ni-PR2 ] Complexes.

The addition of PPh2 H, PPhMeH, PPhH2 , P(para-Tol)H2 , PMesH2 and PH3 to the two-coordinate Ni0 N-heterocyclic carbene species [Ni(NHC)2 ] (NHC=IiPr2 , IMe4 , IEt2 Me2 ) affords a series of mononuclear, terminal phosphido nickel complexes. Structural characterisation of nine of these compounds shows that they have unusual trans [H-Ni-PR2 ] or novel trans [R2 P-Ni-PR2 ] geometries. The bis-phosphido complexes are more accessible when smaller NHCs (IMe4 >IEt2 Me2 >IiPr2 ) and phosphines are employed. P-P activation of the diphosphines R2 P-PR2 (R2 =Ph2 , PhMe) provides an alternative route to some of the [Ni(NHC)2 (PR2 )2 ] complexes. DFT calculations capture these trends with P-H bond activation proceeding from unconventional phosphine adducts in which the H substituent bridges the Ni-P bond. P-P bond activation from [Ni(NHC)2 (Ph2 P-PPh2 )] adducts proceeds with computed barriers below 10 kcal mol-1 . The ability of the [Ni(NHC)2 ] moiety to afford isolable terminal phosphido products reflects the stability of the Ni-NHC bond that prevents ligand dissociation and onward reaction.

Strategic testing approaches for targeted disease monitoring can be used to inform pandemic decision-making.

More than 1.6 million Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) tests were administered daily in the United States at the peak of the epidemic, with a significant focus on individual treatment. Here, we show that objective-driven, strategic sampling designs and analyses can maximize information gain at the population level, which is necessary to increase situational awareness and predict, prepare for, and respond to a pandemic, while also continuing to inform individual treatment. By focusing on specific objectives such as individual treatment or disease prediction and control (e.g., via the collection of population-level statistics to inform lockdown measures or vaccine rollout) and drawing from the literature on capture-recapture methods to deal with nonrandom sampling and testing errors, we illustrate how public health objectives can be achieved even with limited test availability when testing programs are designed a priori to meet those objectives.

Cost and cost-effectiveness of a universal HIV testing and treatment intervention in Zambia and South Africa: evidence and projections from the HPTN 071 (PopART) trial.

BACKGROUND: The HPTN 071 (PopART) trial showed that a combination HIV prevention package including universal HIV testing and treatment (UTT) reduced population-level incidence of HIV compared with standard care. However, evidence is scarce on the costs and cost-effectiveness of such an intervention. METHODS: Using an individual-based model, we simulated the PopART intervention and standard care with antiretroviral therapy (ART) provided according to national guidelines for the 21 trial communities in Zambia and South Africa (for all individuals aged >14 years), with model parameters and primary cost data collected during the PopART trial and from published sources. Two intervention scenarios were modelled: annual rounds of PopART from 2014 to 2030 (PopART 2014-30; as the UNAIDS Fast-Track target year) and three rounds of PopART throughout the trial intervention period (PopART 2014-17). For each country, we calculated incremental cost-effectiveness ratios (ICERs) as the cost per disability-adjusted life-year (DALY) and cost per HIV infection averted. Cost-effectiveness acceptability curves were used to indicate the probability of PopART being cost-effective compared with standard care at different thresholds of cost per DALY averted. We also assessed budget impact by projecting undiscounted costs of the intervention compared with standard care up to 2030. FINDINGS: During 2014-17, the mean cost per person per year of delivering home-based HIV counselling and testing, linkage to care, promotion of ART adherence, and voluntary medical male circumcision via community HIV care providers for the simulated population was US$6·53 (SD 0·29) in Zambia and US$7·93 (0·16) in South Africa. In the PopART 2014-30 scenario, median ICERs for PopART delivered annually until 2030 were $2111 (95% credible interval [CrI] 1827-2462) per HIV infection averted in Zambia and $3248 (2472-3963) per HIV infection averted in South Africa; and $593 (95% CrI 526-674) per DALY averted in Zambia and $645 (538-757) per DALY averted in South Africa. In the PopART 2014-17 scenario, PopART averted one infection at a cost of $1318 (1098-1591) in Zambia and $2236 (1601-2916) in South Africa, and averted one DALY at $258 (225-298) in Zambia and $326 (266-391) in South Africa, when outcomes were projected until 2030. The intervention had almost 100% probability of being cost-effective at thresholds greater than $700 per DALY averted in Zambia, and greater than $800 per DALY averted in South Africa, in the PopART 2014-30 scenario. Incremental programme costs for annual rounds until 2030 were $46·12 million (for a mean of 341 323 people) in Zambia and $30·24 million (for a mean of 165 852 people) in South Africa. INTERPRETATION: Combination prevention with universal home-based testing can be delivered at low annual cost per person but accumulates to a considerable amount when scaled for a growing population. Combination prevention including UTT is cost-effective at thresholds greater than $800 per DALY averted and can be an efficient strategy to reduce HIV incidence in high-prevalence settings. FUNDING: US National Institutes of Health, President's Emergency Plan for AIDS Relief, International Initiative for Impact Evaluation, Bill & Melinda Gates Foundation.

Causes of delayed outbreak responses and their impacts on epidemic spread.

Livestock diseases have devastating consequences economically, socially and politically across the globe. In certain systems, pathogens remain viable after host death, which enables residual transmissions from infected carcasses. Rapid culling and carcass disposal are well-established strategies for stamping out an outbreak and limiting its impact; however, wait-times for these procedures, i.e. response delays, are typically farm-specific and time-varying due to logistical constraints. Failing to incorporate variable response delays in epidemiological models may understate outbreak projections and mislead management decisions. We revisited the 2001 foot-and-mouth epidemic in the United Kingdom and sought to understand how misrepresented response delays can influence model predictions. Survival analysis identified farm size and control demand as key factors that impeded timely culling and disposal activities on individual farms. Using these factors in the context of an existing policy to predict local variation in response times significantly affected predictions at the national scale. Models that assumed fixed, timely responses grossly underestimated epidemic severity and its long-term consequences. As a result, this study demonstrates how general inclusion of response dynamics and recognition of partial controllability of interventions can help inform management priorities during epidemics of livestock diseases.

COVID-19 reopening strategies at the county level in the face of uncertainty: Multiple Models for Outbreak Decision Support.

Policymakers make decisions about COVID-19 management in the face of considerable uncertainty. We convened multiple modeling teams to evaluate reopening strategies for a mid-sized county in the United States, in a novel process designed to fully express scientific uncertainty while reducing linguistic uncertainty and cognitive biases. For the scenarios considered, the consensus from 17 distinct models was that a second outbreak will occur within 6 months of reopening, unless schools and non-essential workplaces remain closed. Up to half the population could be infected with full workplace reopening; non-essential business closures reduced median cumulative infections by 82%. Intermediate reopening interventions identified no win-win situations; there was a trade-off between public health outcomes and duration of workplace closures. Aggregate results captured twice the uncertainty of individual models, providing a more complete expression of risk for decision-making purposes.

OpenABM-Covid19 - an agent-based model for non-pharmaceutical interventions against COVID-19 including contact tracing

SARS-CoV-2 has spread across the world, causing high mortality and unprecedented restrictions on social and economic activity. Policymakers are assessing how best to navigate through the ongoing epidemic, with models being used to predict the spread of infection and assess the impact of public health measures. Here, we present OpenABM-Covid19: an agent-based simulation of the epidemic including detailed age-stratification and realistic social networks. By default the model is parameterised to UK demographics and calibrated to the UK epidemic, however, it can easily be re-parameterised for other countries. OpenABM-Covid19 can evaluate non-pharmaceutical interventions, including both manual and digital contact tracing. It can simulate a population of 1 million people in seconds per day allowing parameter sweeps and formal statistical model-based inference. The code is open-source and has been developed by teams both inside and outside academia, with an emphasis on formal testing, documentation, modularity and transparency. A key feature of OpenABM-Covid19 is its Python interface, which has allowed scientists and policymakers to simulate dynamic packages of interventions and help compare options to suppress the COVID-19 epidemic.