ABSTRACT
IntroductionTo retrospectively assess the accuracy of a mathematical modelling study that projected the rate of COVID-19 diagnoses for 72 locations worldwide in 2021, and to identify predictors of model accuracy. MethodsBetween June and August 2020, an agent-based model was used to project rates of COVID-19 infection incidence and cases diagnosed as positive from 15 September to 31 October 2020 for 72 geographic settings. Five scenarios were modelled: a baseline scenario where no future changes were made to existing restrictions, and four scenarios representing small or moderate changes in restrictions at two intervals. Post hoc, upper and lower bounds for number of diagnosed Covid-19 cases were compared with actual data collected during the prediction window. A regression analysis with 17 covariates was performed to determine correlates of accurate projections. ResultsThe actual data fell within the lower and upper bounds in 27 settings and out of bounds in 45 settings. The only statistically significant predictor of actual data within the predicted bounds was correct assumptions about future policy changes (OR = 15.04; 95%CI 2.20-208.70; p=0.016). ConclusionsFor this study, the accuracy of COVID-19 model projections was dependent on whether assumptions about future policies are correct. Frequent changes in restrictions implemented by governments, which the modelling team was not always able to predict, in part explains why the majority of model projections were inaccurate compared with actual outcomes and supports revision of projections when policies are changed as well as the importance of policy experts collaborating on modelling projects.
ABSTRACT
AimsWe assessed COVID-19 epidemic risks associated with relaxing a set of physical distancing restrictions in the state of Victoria, Australia - a setting with low community transmission - in line with a national framework that aims to balance sequential policy relaxations with longer-term public health and economic need. MethodsAn agent-based model, Covasim, was calibrated to the local COVID-19 epidemiological and policy environment. Contact networks were modelled to capture transmission risks in households, schools and workplaces, and a variety of community spaces (e.g. public transport, parks, bars, cafes/restaurants) and activities (e.g. community or professional sports, large events). Policy changes that could prevent or reduce transmission in specific locations (e.g. opening/closing businesses) were modelled in the context of interventions that included testing, contact tracing (including via a smartphone app), and quarantine. ResultsPolicy changes leading to the gathering of large, unstructured groups with unknown individuals (e.g. bars opening, increased public transport use) posed the greatest risk, while policy changes leading to smaller, structured gatherings with known individuals (e.g. small social gatherings) posed least risk. In the model, epidemic impact following some policy changes took more than two months to occur. Model outcomes support continuation of working from home policies to reduce public transport use, and risk mitigation strategies in the context of social venues opening, such as >30% population-uptake of a contact-tracing app, physical distancing policies within venues reducing transmissibility by >40%, or patron identification records being kept to enable >60% contact tracing. ConclusionsIn a low transmission setting, care should be taken to avoid lifting sequential COVID-19 policy restrictions within short time periods, as it could take more than two months to detect the consequences of any changes. These findings have implications for other settings with low community transmission where governments are beginning to lift restrictions.
