Evaluating the use of social contact data to produce age-specific forecasts of SARS-CoV-2 incidence

Short-term forecasts can provide predictions of how an epidemic will change in the near future and form a central part of outbreak mitigation and control. Renewal-equation based models are increasingly popular. They infer key epidemiological parameters from historical epidemiological data and forecast future epidemic dynamics without requiring complex mechanistic assumptions. However, these models typically ignore interaction between age-groups, partly due to challenges in parameterising a time varying interaction matrix. Social contact data collected regularly by the CoMix survey during the COVID-19 epidemic in England, provide a means to inform interaction between age-groups in real-time. We developed an age-specific forecasting framework and applied it to two age-stratified time-series: incidence of SARS-CoV-2 infection, estimated from a national infection and antibody prevalence survey; and, reported cases according to the UK national COVID-19 dashboard. Jointly fitting our model to social contact data from the CoMix study, we inferred a time-varying next generation matrix which we used to project infections and cases in the four weeks following each of 29 forecast dates between October 2021 and November 2022. We evaluated the forecasts using proper scoring rules and compared performance with three other models with alternative data and specifications alongside two naive baseline models. Overall, incorporating age-interaction improved forecasts of infections and the CoMix-data-informed model was the best performing model at time horizons between two and four weeks. However, this was not true when forecasting cases. We found that age-group-interaction was most important for predicting cases in children and older adults. The contact-data-informed models performed best during the winter months of 2020 - 2021, but performed comparatively poorly in other periods. We highlight challenges regarding the incorporation of contact data in forecasting and offer proposals as to how to extend and adapt our approach, which may lead to more successful forecasts in future.

Evaluating an epidemiologically motivated surrogate model of a multi-model ensemble

Multi-model and multi-team ensemble forecasts have become widely used to generate reliable short-term predictions of infectious disease spread. Notably, various public health agencies have used them to leverage academic disease modelling during the COVID-19 pandemic. However, ensemble forecasts are difficult to interpret and require extensive effort from numerous participating groups as well as a coordination team. In other fields, resource usage has been reduced by training simplified models that reproduce some of the observed behaviour of more complex models. Here we used observations of the behaviour of the European COVID-19 Forecast Hub ensemble combined with our own forecasting experience to identify a set of properties present in current ensemble forecasts. We then developed a parsimonious forecast model intending to mirror these properties. We assess forecasts generated from this model in real time over six months (the 15th of January 2022 to the 19th of July 2022) and for multiple European countries. We focused on forecasts of cases one to four weeks ahead and compared them to those by the European forecast hub ensemble. We find that the surrogate model behaves qualitatively similarly to the ensemble in many instances, though with increased uncertainty and poorer performance around periods of peak incidence (as measured by the Weighted Interval Score). The performance differences, however, seem to be partially due to a subset of time points, and the proposed model appears better probabilistically calibrated than the ensemble. We conclude that our simplified forecast model may have captured some of the dynamics of the hub ensemble, but more work is needed to understand the implicit epidemiological model that it represents.

Inferring the differences in incubation-period and generation-interval distributions of the Delta and Omicron variants of SARS-CoV-2

Estimating the differences in the incubation-period, serial-interval, and generation-interval distributions of SARS-CoV-2 variants is critical to understanding their transmission and control. However, the impact of epidemic dynamics is often neglected in estimating the timing of infection and transmission--for example, when an epidemic is growing exponentially, a cohort of infected individuals who developed symptoms at the same time are more likely to have been infected recently. Here, we re-analyze incubation-period and serial-interval data describing transmissions of the Delta and Omicron variants from the Netherlands at the end of December 2021. Previous analysis of the same data set reported shorter mean observed incubation period (3.2 days vs 4.4 days) and serial interval (3.5 days vs 4.1 days) for the Omicron variant, but the number of infections caused by the Delta variant decreased during this period as the number of Omicron infections increased. When we account for growth-rate differences of two variants during the study period, we estimate similar mean incubation periods (3.8-4.5 days) for both variants but a shorter mean generation interval for the Omicron variant (3.0 days; 95% CI: 2.7-3.2 days) than for the Delta variant (3.8 days; 95% CI: 3.7-4.0 days). We further note that the differences in estimated generation intervals may be driven by the "network effect"--higher effective transmissibility of the Omicron variant can cause faster susceptible depletion among contact networks, which in turn prevents late transmission (therefore shortening realized generation intervals). Using up-to-date generation-interval distributions is critical to accurately estimating the reproduction advantage of the Omicron variant. SignificanceRecent studies suggest that individuals infected with the Omicron variant develop symptoms earlier (shorter incubation period) and transmit faster (shorter generation interval) than those infected with the Delta variant. However, these studies typically neglect population-level effects: when an epidemic is growing, a greater proportion of current cases were infected recently, biasing us toward observing faster transmission events. Accounting for this dynamical bias, we find that Omicron infections from the Netherlands at the end of December 2021 had similar incubation periods, but shorter generation intervals, compared to Delta infections from the same period. Shorter generation intervals of the Omicron variant might be due to its higher effective reproduction number, which can cause faster local susceptible depletion around the contact network.

Cycle threshold values in symptomatic COVID-19 cases in England

IntroductionSince the start of the pandemic SARS-CoV-2 infection has most commonly been confirmed using reverse transcriptase polymerase chain reaction (RT-PCR), with results translated into a binary positive/negative outcomes. Previous studies have found that there is additional useful information in the level of the Cycle threshold (Ct value) of positive cases. Here we characterise variation in Ct values as a proxy for viral loads in more than 3 million test-positive COVID-19 cases in England with the aim of better quantifying the utility of such data. MethodsWe used individual N gene Ct values from symptomatic PCR positive (with Ct value less than 30) Pillar 2 cases in England who self-reported the date of symptom onset, and for whom age, reinfection status, variant status, and the number of vaccines received was available. Those with a positive test result more than 6 days after their reported symptom onset were excluded to mitigate the potential impact of recall bias. We used a generalised additive model, to estimate Ct values empirical mean Ct values for each strata of interest independently as well as to predict Ct values using a model that adjusted for a range of demographic and epidemiological covariates jointly. We present empirical Ct values and compare them to predicted mean Ct values. ResultsWe found that mean Ct values varied by vaccine status, and reinfection status with the number of vaccine doses having little apparent effect. Modelling Ct values as a smooth function of time since onset and other variables struggled to reproduce the individual variation in the data but did match the population-level variation over time relatively well with this being apparently dominated by large differences between variants. Other variation over time was also captured to some degree though their remained several periods where the model could not capture the empirical means with a potential explanation being epidemic phase bias. ConclusionsAnalysing a large dataset of routine Ct values from symptomatic COVID-19 cases in England we found variation based on time since symptom onset, vaccine status, age, and variant. Ct values were highest 1-3 days after symptom onset and differed most due to variant status. We found no clear correlation between previously estimated differences in intrinsic transmissibility and Ct values indicating that this is potentially mediated at least partly by factors other than viral load as estimated using Ct values. We found evidence that a model adjusting for a range of covariates could explain some of the population-level variation over time but systematically underestimated Ct values when incidence was increasing, and overestimated them when incidence was decreasing. This indicates the utility of Ct values from this data source as a tool for surveillance, potentially avoiding some of the biases of aggregated positive counts.

Predictive performance of multi-model ensemble forecasts of COVID-19 across European nations

BackgroundShort-term forecasts of infectious disease burden can contribute to situational awareness and aid capacity planning. Based on best practice in other fields and recent insights in infectious disease epidemiology, one can maximise the predictive performance of such forecasts if multiple models are combined into an ensemble. Here we report on the performance of ensembles in predicting COVID-19 cases and deaths across Europe between 08 March 2021 and 07 March 2022. MethodsWe used open-source tools to develop a public European COVID-19 Forecast Hub. We invited groups globally to contribute weekly forecasts for COVID-19 cases and deaths reported from a standardised source over the next one to four weeks. Teams submitted forecasts from March 2021 using standardised quantiles of the predictive distribution. Each week we created an ensemble forecast, where each predictive quantile was calculated as the equally-weighted average (initially the mean and then from 26th July the median) of all individual models predictive quantiles. We measured the performance of each model using the relative Weighted Interval Score (WIS), comparing models forecast accuracy relative to all other models. We retrospectively explored alternative methods for ensemble forecasts, including weighted averages based on models past predictive performance. ResultsOver 52 weeks we collected and combined up to 28 forecast models for 32 countries. We found a weekly ensemble had a consistently strong performance across countries over time. Across all horizons and locations, the ensemble performed better on relative WIS than 84% of participating models forecasts of incident cases (with a total N=862), and 92% of participating models forecasts of deaths (N=746). Across a one to four week time horizon, ensemble performance declined with longer forecast periods when forecasting cases, but remained stable over four weeks for incident death forecasts. In every forecast across 32 countries, the ensemble outperformed most contributing models when forecasting either cases or deaths, frequently outperforming all of its individual component models. Among several choices of ensemble methods we found that the most influential and best choice was to use a median average of models instead of using the mean, regardless of methods of weighting component forecast models. ConclusionsOur results support the use of combining forecasts from individual models into an ensemble in order to improve predictive performance across epidemiological targets and populations during infectious disease epidemics. Our findings further suggest that median ensemble methods yield better predictive performance more than ones based on means. Our findings also highlight that forecast consumers should place more weight on incident death forecasts than incident case forecasts at forecast horizons greater than two weeks. Code and data availabilityAll data and code are publicly available on Github: covid19-forecast-hub-europe/euro-hub-ensemble.

Estimating epidemiological quantities from repeated cross-sectional prevalence measurements

BackgroundRepeated measurements of cross-sectional prevalence of Polymerase Chain Reaction (PCR) positivity or seropositivity provide rich insight into the dynamics of an infection. The UK Office for National Statistics (ONS) Community Infection Survey publishes such measurements for SARS-CoV-2 on a weekly basis based on testing enrolled households, contributing to situational awareness in the country. Here we present estimates of time-varying and static epidemiological quantities that were derived from the estimates published by ONS. MethodsWe used a gaussian process to model incidence of infections and then estimated observed PCR prevalence by convolving our modelled incidence estimates with a previously published PCR detection curve describing the probability of a positive test as a function of the time since infection. We refined our incidence estimates using time-varying estimates of antibody prevalence combined with a model of antibody positivity and waning that moved individuals between compartments with or without antibodies based on estimates of new infections, vaccination, probability of seroconversion and waning. ResultsWe produced incidence curves of infection describing the UK epidemic from late April 2020 until early 2022. We used these estimates of incidence to estimate the time-varying growth rate of infections, and combined them with estimates of the generation interval to estimate time-varying reproduction numbers. Biological parameters describing seroconversion and waning, while based on a simple model, were broadly in line with plausible ranges from individual-level studies. ConclusionsBeyond informing situational awareness and allowing for estimates using individual-level data, repeated cross-sectional studies make it possible to estimate epidemiological parameters from population-level models. Studies or public health surveillance methods based on similar designs offer opportunities for further improving our understanding of the dynamics of SARS-CoV-2 or other pathogens and their interaction with population-level immunity.

Estimation of the test to test distribution as a proxy for generation interval distribution for the Omicron variant in England

BackgroundEarly estimates from South Africa indicated that the Omicron COVID-19 variant may be both more transmissible and have greater immune escape than the previously dominant Delta variant. The rapid turnover of the latest epidemic wave in South Africa as well as initial evidence from contact tracing and household infection studies has prompted speculation that the generation time of the Omicron variant may be shorter in comparable settings than the generation time of the Delta variant. MethodsWe estimated daily growth rates for the Omicron and Delta variants in each UKHSA region from the 23rd of November to the 23rd of December 2021 using surveillance case counts by date of specimen and S-gene target failure status with an autoregressive model that allowed for time-varying differences in the transmission advantage of the Delta variant where the evidence supported this. By assuming a gamma distributed generation distribution we then estimated the generation time distribution and transmission advantage of the Omicron variant that would be required to explain this time varying advantage. We repeated this estimation process using two different prior estimates for the generation time of the Delta variant first based on household transmission and then based on its intrinsic generation time. ResultsVisualising our growth rate estimates provided initial evidence for a difference in generation time distributions. Assuming a generation time distribution for Delta with a mean of 2.5-4 days (90% credible interval) and a standard deviation of 1.9-3 days we estimated a shorter generation time distribution for Omicron with a mean of 1.5-3.2 days and a standard deviation of 1.3-4.6 days. This implied a transmission advantage for Omicron in this setting of 160%-210% compared to Delta. We found similar relative results using an estimate of the intrinsic generation time for Delta though all estimates increased in magnitude due to the longer assumed generation time. ConclusionsWe found that a reduction in the generation time of Omicron compared to Delta was able to explain the observed variation over time in the transmission advantage of the Omicron variant. However, this analysis cannot rule out the role of other factors such as differences in the populations the variants were mixing in, differences in immune escape between variants or bias due to using the test to test distribution as a proxy for the generation time distribution.

Comparing human and model-based forecasts of COVID-19 in Germany and Poland

1Forecasts based on epidemiological modelling have played an important role in shaping public policy throughout the COVID-19 pandemic. This modelling combines knowledge about infectious disease dynamics with the subjective opinion of the researcher who develops and refines the model and often also adjusts model outputs. Developing a forecast model is difficult, resource- and time-consuming. It is therefore worth asking what modelling is able to add beyond the subjective opinion of the researcher alone. To investigate this, we analysed different real-time forecasts of cases of and deaths from COVID-19 in Germany and Poland over a 1-4 week horizon submitted to the German and Polish Forecast Hub. We compared crowd forecasts elicited from researchers and volunteers, against a) forecasts from two semi-mechanistic models based on common epidemiological assumptions and b) the ensemble of all other models submitted to the Forecast Hub. We found crowd forecasts, despite being overconfident, to outperform all other methods across all forecast horizons when forecasting cases (weighted interval score relative to the Hub ensemble 2 weeks ahead: 0.89). Forecasts based on computational models performed comparably better when predicting deaths (rel. WIS 1.26), suggesting that epidemiological modelling and human judgement can complement each other in important ways.

Estimating the increase in reproduction number associated with the Delta variant using local area dynamics in England

BackgroundLocal estimates of the time-varying effective reproduction number (Rt) of COVID-19 in England became increasingly heterogeneous during April and May 2021. This may have been attributable to the spread of the Delta SARS-CoV-2 variant. This paper documents real-time analysis that aimed to investigate the association between changes in the proportion of positive cases that were S-gene positive, an indicator of the Delta variant against a background of the previously predominant Alpha variant, and the estimated time-varying Rt at the level of upper-tier local authorities (UTLA). MethodWe explored the relationship between the proportion of samples that were S-gene positive and the Rt of test-positive cases over time from the 23 February 2021 to the 25 May 2021. Effective reproduction numbers were estimated using the EpiNow2 R package independently for each local authority using two different estimates of the generation time. We then fit a range of regression models to estimate a multiplicative relationship between S-gene positivity and weekly mean Rt estimate. ResultsWe found evidence of an association between increased mean Rt estimates and the proportion of S-gene positives across all models evaluated with the magnitude of the effect increasing as model flexibility was decreased. Models that adjusted for either national level or NHS region level time-varying residuals were found to fit the data better, suggesting potential unexplained confounding. ConclusionsOur results indicated that even after adjusting for time-varying residuals between NHS regions, S-gene positivity was associated with an increase in the effective reproduction number of COVID-19. These findings were robust across a range of models and generation time assumptions, though the specific effect size was variable depending on the assumptions used. The lower bound of the estimated effect indicated that the reproduction number of Delta was above 1 in almost all local authorities throughout the period of investigation.

Unexposed populations and potential COVID-19 burden in European countries

We estimate the potential remaining COVID-19 burden in 19 European countries by estimating the proportion of each countrys population that has acquired immunity to severe disease through infection or vaccination. Our results suggest that many European countries could still face a substantial burden of hospitalisations and deaths, particularly those with lower vaccination coverage, less historical transmission, and/or older populations. Continued non-pharmaceutical interventions and efforts to achieve high vaccination coverage are required in these countries to limit severe COVID-19 outcomes.

National and subnational short-term forecasting of COVID-19 in Germany and Poland, early 2021

BackgroundDuring the COVID-19 pandemic there has been a strong interest in forecasts of the short-term development of epidemiological indicators to inform decision makers. In this study we evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland for the period from January through April 2021. MethodsWe evaluate probabilistic real-time predictions of confirmed cases and deaths from COVID-19 in Germany and Poland. These were issued by 15 different forecasting models, run by independent research teams. Moreover, we study the performance of combined ensemble forecasts. Evaluation of probabilistic forecasts is based on proper scoring rules, along with interval coverage proportions to assess forecast calibration. The presented work is part of a pre-registered evaluation study and covers the period from January through April 2021. ResultsWe find that many, though not all, models outperform a simple baseline model up to four weeks ahead for the considered targets. Ensemble methods (i.e., combinations of different available forecasts) show very good relative performance. The addressed time period is characterized by rather stable non-pharmaceutical interventions in both countries, making short-term predictions more straightforward than in previous periods. However, major trend changes in reported cases, like the rebound in cases due to the rise of the B.1.1.7 (alpha) variant in March 2021, prove challenging to predict. ConclusionsMulti-model approaches can help to improve the performance of epidemiological forecasts. However, while death numbers can be predicted with some success based on current case and hospitalization data, predictability of case numbers remains low beyond quite short time horizons. Additional data sources including sequencing and mobility data, which were not extensively used in the present study, may help to improve performance. Plain language summaryThe goal of this study is to assess the quality of forecasts of weekly case and death numbers of COVID-19 in Germany and Poland during the period of January through April 2021. We focus on real-time forecasts at time horizons of one and two weeks ahead created by fourteen independent teams. Forecasts are systematically evaluated taking uncertainty ranges of predictions into account. We find that combining different forecasts into ensembles can improve the quality of predictions, but especially case numbers proved very challenging to predict beyond quite short time windows. Additional data sources, in particular genetic sequencing data, may help to improve forecasts in the future.

Estimating the relationship between mobility, non-pharmaceutical interventions, and COVID-19 transmission in Ghana

BackgroundGovernments around the world have implemented non-pharmaceutical interventions to limit the transmission of COVID-19. While lockdowns and physical distancing have proven effective for reducing COVID-19 transmission, there is still limited understanding of how NPI measures are reflected in indicators of human mobility. Further, there is a lack of understanding about how findings from high-income settings correspond to low and middle-income contexts. MethodsIn this study, we assess the relationship between indicators of human mobility, NPIs, and estimates of Rt, a real-time measure of the intensity of COVID-19 transmission. We construct a multilevel generalised linear mixed model, combining local disease surveillance data from subnational districts of Ghana with the timing of NPIs and indicators of human mobility from Google and Vodafone Ghana. FindingsWe observe a relationship between reductions in human mobility and decreases in Rt during the early stages of the COVID-19 epidemic in Ghana. We find that the strength of this relationship varies through time, decreasing after the most stringent period of interventions in the early epidemic. InterpretationOur findings demonstrate how the association of NPI and mobility indicators with COVID-19 transmission may vary through time. Further, we demonstrate the utility of combining local disease surveillance data with large scale human mobility data to augment existing surveillance capacity and monitor the impact of NPI policies. Research in ContextO_ST_ABSEvidence before this studyC_ST_ABSWe searched PubMed and preprint archives for articles published in English that contained information about the COVID-19 pandemic published up to Nov 1, 2021, using the search terms "coronavirus", "CoV", "COVID-19", "mobility", "movement", and "flow". The data thus far suggests that NPI measures including physical distancing, reduction of travel, and use of personal protective equipment have been demonstrated to reduce COVID-19 transmission. Much of the existing research focuses on comparisons of NPI stringency with COVID-19 transmission among different high-income countries, or on high-income countries, leaving critical questions about the applicability of these findings to low- and middle-income settings. Added value of this studyWe used a detailed COVID-19 surveillance dataset from Ghana, and unique high resolution spatial data on human mobility from Vodafone Ghana as well as Google smartphone GPS location data. We show how human mobility and NPI stringency were associated with changes in the effective reproduction number (Rt). We further demonstrate how this association was strongest in the early COVID-19 outbreak in Ghana, decreasing after the relaxation of national restrictions. Implications of all the available evidenceThe change in association between human mobility, NPI stringency, and Rt may reflect a "decoupling" of NPI stringency and human mobility from disease transmission in Ghana as the COVID-19 epidemic progressed. This finding provides public health decision makers with important insights for the understanding of the utility of mobility data for predicting the spread of COVID-19.

Comparative assessment of methods for short-term forecasts of COVID-19 admissions in England at the local level

BackgroundForecasting healthcare demand is essential in epidemic settings, both to inform situational awareness and facilitate resource planning. Ideally, forecasts should be robust across time and locations. During the COVID-19 pandemic in England, it is an ongoing concern that demand for hospital care for COVID-19 patients in England will exceed available resources. MethodsWe made weekly forecasts of daily COVID-19 hospital admissions for National Health Service (NHS) Trusts in England between August 2020 and April 2021 using three disease-agnostic forecasting models: a mean ensemble of autoregressive time series models, a linear regression model with 7-day-lagged local cases as a predictor, and a scaled convolution of local cases and a delay distribution. We compared their point and probabilistic accuracy to a mean-ensemble of them all, and to a simple baseline model of no change from the last day of admissions. We measured predictive performance using the Weighted Interval Score (WIS) and considered how this changed in different scenarios (the length of the predictive horizon, the date on which the forecast was made, and by location), as well as how much admissions forecasts improved when future cases were known. ResultsAll models outperformed the baseline in the majority of scenarios. Forecasting accuracy varied by forecast date and location, depending on the trajectory of the outbreak, and all individual models had instances where they were the top- or bottom-ranked model. Forecasts produced by the mean-ensemble were both the most accurate and most consistently accurate forecasts amongst all the models considered. Forecasting accuracy was improved when using future observed, rather than forecast, cases, especially at longer forecast horizons. ConclusionsAssuming no change in current admissions is rarely better than including at least a trend. Using confirmed COVID-19 cases as a predictor can improve admissions forecasts in some scenarios, but this is variable and depends on the ability to make consistently good case forecasts. However, ensemble forecasts can make forecasts that make consistently more accurate forecasts across time and locations. Given minimal requirements on data and computation, our admissions forecasting ensemble could be used to anticipate healthcare needs in future epidemic or pandemic settings.

Tracking the emergence of disparities in the subnational spread of COVID-19 in Brazil using an online application for real-time data visualisation.

BackgroundBrazil is one of the countries worst affected by the COVID-19 pandemic with over 20 million cases and 557,000 deaths reported. Comparison of real-time local COVID-19 data between areas is essential for understanding transmission, measuring the effects of interventions and predicting the course of the epidemic, but are often challenging due to different population sizes and structures. MethodsWe describe the development of a new app for the real-time visualisation of COVID-19 data in Brazil at the municipality level. In the CLIC-Brazil app, daily updates of case and death data are downloaded, age standardised and used to estimate reproduction number (Rt). We show how such platforms can perform real-time regression analyses to identify factors associated with the rate of initial spread and early reproduction number. We also use survival methods to predict the likelihood of occurrence of a new peak of COVID-19 incidence. FindingsAfter an initial introduction in Sao Paulo and Rio de Janeiro states in early March 2020, the epidemic spread to Northern states and then to highly populated coastal regions and the Central-West. Municipalities with higher metrics of social development experienced earlier arrival of COVID-19 (decrease of 11{middle dot}1 days [95% CI:13{middle dot}2,8{middle dot}9] in the time to arrival for each 10% increase in the social development index). Differences in the initial epidemic intensity (mean Rt) were largely driven by geographic location and the date of local onset. InterpretationThis study demonstrates that platforms that monitor, standardise and analyse the epidemiological data at a local level can give useful real-time insights into outbreak dynamics that can be used to better adapt responses to the current and future pandemics. FundingThis project was supported by a Medical Research Council UK (MRC-UK) -Sao Paulo Research Foundation (FAPESP) CADDE partnership award (MR/S0195/1 and FAPESP 18/14389-0)

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multi-model ensemble forecast that combined predictions from dozens of different research groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naive baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-week horizon 3-5 times larger than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. Significance StatementThis paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the US. Results show high variation in accuracy between and within stand-alone models, and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public health action.

Short-term forecasting of COVID-19 in Germany and Poland during the second wave - a preregistered study

We report insights from ten weeks of collaborative COVID-19 forecasting for Germany and Poland (12 October - 19 December 2020). The study period covers the onset of the second wave in both countries, with tightening non-pharmaceutical interventions (NPIs) and subsequently a decay (Poland) or plateau and renewed increase (Germany) in reported cases. Thirteen independent teams provided probabilistic real-time forecasts of COVID-19 cases and deaths. These were reported for lead times of one to four weeks, with evaluation focused on one- and two-week horizons, which are less affected by changing NPIs. Heterogeneity between forecasts was considerable both in terms of point predictions and forecast spread. Ensemble forecasts showed good relative performance, in particular in terms of coverage, but did not clearly dominate single-model predictions. The study was preregistered and will be followed up in future phases of the pandemic.

Estimated transmissibility and severity of novel SARS-CoV-2 Variant of Concern 202012/01 in England

A novel SARS-CoV-2 variant, VOC 202012/01 (lineage B.1.1.7), emerged in southeast England in November 2020 and is rapidly spreading towards fixation. Using a variety of statistical and dynamic modelling approaches, we estimate that this variant has a 43-90% (range of 95% credible intervals 38-130%) higher reproduction number than preexisting variants. A fitted two-strain dynamic transmission model shows that VOC 202012/01 will lead to large resurgences of COVID-19 cases. Without stringent control measures, including limited closure of educational institutions and a greatly accelerated vaccine roll-out, COVID-19 hospitalisations and deaths across England in 2021 will exceed those in 2020. Concerningly, VOC 202012/01 has spread globally and exhibits a similar transmission increase (59-74%) in Denmark, Switzerland, and the United States.

The effectiveness of population-wide, rapid antigen test based screening in reducing SARS-CoV-2 infection prevalence in Slovakia

Slovakia conducted multiple rounds of population-wide rapid antigen testing for SARS-CoV-2 in late 2020, combined with a period of additional contact restrictions. Observed prevalence decreased by 58% (95% CI: 57-58%) within one week in the 45 counties that were subject to two rounds of mass testing, an estimate that remained robust when adjusting for multiple potential confounders. Adjusting for epidemic growth of 4.4% (1.1-6.9%) per day preceding the mass testing campaign, the estimated decrease in prevalence compared to a scenario of unmitigated growth was 70% (67-73%). Modelling suggests that this decrease cannot be explained solely by infection control measures, but requires the additional impact of isolation as well as quarantine of household members of those testing positive.

Short-term forecasts to inform the response to the COVID-19 epidemic in the UK

BackgroundShort-term forecasts of infectious disease can aid situational awareness and planning for outbreak response. Here, we report on multi-model forecasts of Covid-19 in the UK that were generated at regular intervals starting at the end of March 2020, in order to monitor expected healthcare utilisation and population impacts in real time. MethodsWe evaluated the performance of individual model forecasts generated between 24 March and 14 July 2020, using a variety of metrics including the weighted interval score as well as metrics that assess the calibration, sharpness, bias and absolute error of forecasts separately. We further combined the predictions from individual models into ensemble forecasts using a simple mean as well as a quantile regression average that aimed to maximise performance. We compared model performance to a null model of no change. ResultsIn most cases, individual models performed better than the null model, and ensembles models were well calibrated and performed comparatively to the best individual models. The quantile regression average did not noticeably outperform the mean ensemble. ConclusionsEnsembles of multi-model forecasts can inform the policy response to the Covid-19 pandemic by assessing future resource needs and expected population impact of morbidity and mortality.

Evaluating the use of the reproduction number as an epidemiological tool, using spatio-temporal trends of the Covid-19 outbreak in England

The time-varying reproduction number (Rt: the average number secondary infections caused by each infected person) may be used to assess changes in transmission potential during an epidemic. While new infections are not usually observed directly, they can be estimated from data. However, data may be delayed and potentially biased. We investigated the sensitivity of Rt estimates to different data sources representing Covid-19 in England, and we explored how this sensitivity could track epidemic dynamics in population sub-groups. We sourced public data on test-positive cases, hospital admissions, and deaths with confirmed Covid-19 in seven regions of England over March through August 2020. We estimated Rt using a model that mapped unobserved infections to each data source. We then compared differences in Rt with the demographic and social context of surveillance data over time. Our estimates of transmission potential varied for each data source, with the relative inconsistency of estimates varying across regions and over time. Rt estimates based on hospital admissions and deaths were more spatio-temporally synchronous than when compared to estimates from all test-positives. We found these differences may be linked to biased representations of subpopulations in each data source. These included spatially clustered testing, and where outbreaks in hospitals, care homes, and young age groups reflected the link between age and severity of disease. We highlight that policy makers could better target interventions by considering the source populations of Rt estimates. Further work should clarify the best way to combine and interpret Rt estimates from different data sources based on the desired use.