Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multi-model ensemble forecast that combined predictions from dozens of different research groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naive baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-week horizon 3-5 times larger than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. Significance StatementThis paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the US. Results show high variation in accuracy between and within stand-alone models, and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public health action.

Household Transmission of SARS-COV-2: Insights from a Population-based Serological Survey

BackgroundKnowing the transmissibility of asymptomatic infections and risk of infection from household- and community-exposures is critical to SARS-CoV-2 control. Limited previous evidence is based primarily on virologic testing, which disproportionately misses mild and asymptomatic infections. Serologic measures are more likely to capture all previously infected individuals. ObjectiveEstimate the risk of SARS-CoV-2 infection from household and community exposures, and identify key risk factors for transmission and infection. DesignCross-sectional household serosurvey and transmission model. SettingGeneva, Switzerland Participants4,524 household members [≥]5 years from 2,267 households enrolled April-June 2020. MeasurementsPast SARS-CoV-2 infection confirmed through IgG ELISA. Chain-binomial models based on the number of infections within households used to estimate the cumulative extra-household infection risk and infection risk from exposure to an infected household member by demographics and infectors symptoms. ResultsThe chance of being infected by a SARS-CoV-2 infected household member was 17.3% (95%CrI,13.7-21.7%) compared to a cumulative extra-household infection risk of 5.1% (95%CrI,4.5-5.8%). Infection risk from an infected household member increased with age, with 5-9 year olds having 0.4 times (95%CrI, 0.07-1.4) the odds of infection, and [≥]65 years olds having 2.7 (95%CrI,0.88-7.4) times the odds of infection of 20-49 year olds. Working-age adults had the highest extra-household infection risk. Seropositive asymptomatic household members had 69.6% lower odds (95%CrI,33.7-88.1%) of infecting another household member compared to those reporting symptoms, accounting for 14.7% (95%CrI,6.3-23.2%) of all household infections. LimitationsSelf-reported symptoms, small number of seropositive kids and imperfect serologic tests. ConclusionThe risk of infection from exposure to a single infected household member was more than three-times that of extra-household exposures over the first pandemic wave. Young children had a lower risk of infection from household members. Asymptomatic infections are far less likely to transmit than symptomatic ones but do cause infections. Funding SourceSwiss Federal Office of Public Health, Swiss School of Public Health (Corona Immunitas research program), Fondation de Bienfaisance du Groupe Pictet, Fondation Ancrage, Fondation Privee des Hopitaux Universitaires de Geneve, and Center for Emerging Viral Diseases.

Dynamics and significance of the antibody response to SARS-CoV-2 infection

BACKGROUNDCharacterizing the humoral immune response to SARS-CoV-2 and developing accurate serologic assays are needed for diagnostic purposes and estimating population-level seroprevalence. METHODSWe measured the kinetics of early antibody responses to the receptor-binding domain (RBD) of the spike (S) protein of SARS-CoV-2 in a cohort of 259 symptomatic North American patients infected with SARS-CoV-2 (up to 75 days after symptom onset) compared to antibody levels in 1548 individuals whose blood samples were obtained prior to the pandemic. RESULTSBetween 14-28 days from onset of symptoms, IgG, IgA, or IgM antibody responses to RBD were all accurate in identifying recently infected individuals, with 100% specificity and a sensitivity of 97%, 91%, and 81% respectively. Although the estimated median time to becoming seropositive was similar across isotypes, IgA and IgM antibodies against RBD were short-lived with most individuals estimated to become seronegative again by 51 and 47 days after symptom onset, respectively. IgG antibodies against RBD lasted longer and persisted through 75 days post-symptoms. IgG antibodies to SARS-CoV-2 RBD were highly correlated with neutralizing antibodies targeting the S protein. No cross-reactivity of the SARS-CoV-2 RBD-targeted antibodies was observed with several known circulating coronaviruses, HKU1, OC 229 E, OC43, and NL63. CONCLUSIONSAmong symptomatic SARS-CoV-2 cases, RBD-targeted antibodies can be indicative of previous and recent infection. IgG antibodies are correlated with neutralizing antibodies and are possibly a correlate of protective immunity.

Serology-informed estimates of SARS-COV-2 infection fatality risk in Geneva, Switzerland

The infection fatality risk (IFR) is the average number of deaths per infection by a pathogen and is key to characterizing the severity of infection across the population and for specific demographic groups. To date, there are few empirical estimates of IFR published due to challenges in measuring infection rates.1,2 Outside of closed, closely surveilled populations where infection rates can be monitored through viral surveillance, we must rely on indirect measures of infection, like specific antibodies. Representative seroprevalence studies provide an important avenue for estimating the number of infections in a community, and when combined with death counts can lead to robust estimates of the IFR.

A scenario modeling pipeline for COVID-19 emergency planning

Coronavirus disease 2019 (COVID-19) has caused strain on health systems worldwide due to its high mortality rate and the large portion of cases requiring critical care and mechanical ventilation. During these uncertain times, public health decision makers, from city health departments to federal agencies, sought the use of epidemiological models for decision support in allocating resources, developing non-pharmaceutical interventions, and characterizing the dynamics of COVID-19 in their jurisdictions. In response, we developed a flexible scenario modeling pipeline that could quickly tailor models for decision makers seeking to compare projections of epidemic trajectories and healthcare impacts from multiple intervention scenarios in different locations. Here, we present the components and configurable features of the COVID Scenario Pipeline, with a vignette detailing its current use. We also present model limitations and active areas of development to meet ever-changing decision maker needs.

Repeated seroprevalence of anti-SARS-CoV-2 IgG antibodies in a population-based sample from Geneva, Switzerland

BackgroundAssessing the burden of COVID-19 based on medically-attended case counts is suboptimal given its reliance on testing strategy, changing case definitions and the wide spectrum of disease presentation. Population-based serosurveys provide one avenue for estimating infection rates and monitoring the progression of the epidemic, overcoming many of these limitations. MethodsTaking advantage of a pool of adult participants from population-representative surveys conducted in Geneva, Switzerland, we implemented a study consisting of 8 weekly serosurveys among these participants and their household members older than 5 years. We tested each participant for anti-SARS-CoV-2-IgG antibodies using a commercially available enzyme-linked immunosorbent assay (Euroimmun AG, Lubeck, Germany). We estimated seroprevalence using a Bayesian regression model taking into account test performance and adjusting for the age and sex of Genevas population. ResultsIn the first three weeks, we enrolled 1335 participants coming from 633 households, with 16% <20 years of age and 53.6% female, a distribution similar to that of Geneva. In the first week, we estimated a seroprevalence of 3.1% (95% CI 0.2-5.99, n=343). This increased to 6.1% (95% CI 2.69.33, n=416) in the second, and to 9.7% (95% CI 6.1-13.11, n=576) in the third week. We found that 5-19 year-olds (6.0%, 95% CI 2.3-10.2%) had similar seroprevalence to 20-49 year olds (8.5%, 95%CI 4.99-11.7), while significantly lower seroprevalence was observed among those 50 and older (3.7%, 95% CI 0.99-6.0, p=0.0008). InterpretationAssuming that the presence of IgG antibodies is at least in the short-term associated with immunity, these results highlight that the epidemic is far from burning out simply due to herd immunity. Further, no differences in seroprevalence between children and middle age adults are observed. These results must be considered as Switzerland and the world look towards easing restrictions aimed at curbing transmission.

COVID-19: Projecting the impact in Rohingya refugee camps and beyond

BackgroundCOVID-19 could have even more dire consequences in refugees camps than in general populations. Bangladesh has confirmed COVID-19 cases and hosts almost 1 million Rohingya refugees from Myanmar with 600,000 concentrated in Kutupalong-Balukhali Expansion Site (age mean: 21 years, sd: 18 years, 52% female). Projections of the potential COVID-19 burden, epidemic speed, and healthcare needs in such settings are critical for preparedness planning. Methods and FindingsTo explore the potential impact of the introduction of SARS-CoV-2 in Kutupalong-Balukhali Expansion Site, we used a stochastic SEIR transmission model with parameters derived from emerging literature and age as the primary determinant of infection severity. We considered three scenarios with different assumptions about the transmission potential of SARS-CoV-2. From the simulated infections, we estimated hospitalizations, deaths, and healthcare needs expected, age-adjusted for the Kutupalong-Balukhali Expansion Site age distribution. Our findings suggest that a large-scale outbreak is likely after a single introduction of the virus into the camp with 61-92% of simulations leading to at least 1,000 people infected across scenarios. On average, in the first 30 days of the outbreak, we expect 18 (95% prediction interval (PI), 2-65), 54 (95% PI, 3-223), and 370 (95% PI, 4-1,850) people infected in the low, moderate, and high transmission scenarios, respectively. These reach 421,500 (95% PI, 376,300-463,500), 546,800 (95% PI, 499,300-567,000) and 589,800 (95% PI, 578,800-595,600) people infected in 12 months, respectively. Hospitalization needs exceeded the existing hospitalization capacity of 340 beds after 55-136 days between the low and high transmission scenarios. We estimate 2,040 (95% PI, 1,660-2,500), 2,650 (95% PI, 2,030-3,380), and 2,880 (95% PI, 2,090-3,830) deaths in the low, moderate and high transmission scenarios, respectively. Due to limited data at the time of analyses, we assumed that age was the primary determinant of infection severity and hospitalization. We expect that comorbidities and limited hospitalization and intensive care capacity may increase this risk, thus we may be underestimating the potential burden. ConclusionsOur findings suggest that a COVID-19 epidemic in a refugee settlement may have profound consequences, requiring large increases in healthcare capacity and infrastructure that may exceed what is currently feasible in these settings. Detailed and realistic planning for the worst-case in Kutupalong-Balukhali and all refugee camps worldwide must begin now. Plans should consider novel and radical strategies to reduce infectious contacts and fill health worker gaps while recognizing that refugees may not have access to national health systems. AUTHORS SUMMARYO_LSTWhy was this study done?C_LSTO_LIForcibly displaced populations, especially those who reside in settlements with high density, poor access to water and sanitation, and limited health services, are especially vulnerable to COVID-19. C_LIO_LIBangladesh, which has confirmed COVID-19 cases, hosts almost 900,000 Rohingya refugees from Myanmar in the Coxs Bazar district, approximately 600,000 of whom are concentrated in the Kutupalong-Balukhali Expansion Site. C_LIO_LIThe capacity to meet the existing health needs of this population is limited; an outbreak of COVID-19 within this population threatens to severely disrupt an already fragile situation. C_LIO_LIWe conducted this study to estimate the number of people infected, hospitalizations, and deaths that might occur in the Kutupalong-Balukhali Expansion Site to inform ongoing preparedness and response activities by the Bangladesh government, the United Nations agencies, and other national and international actors. C_LI O_LSTWhat did the researchers do and find?C_LSTO_LIUsing a dynamic model of SARS-CoV-2 transmission, we simulated how a COVID-19 outbreak could spread within the Expansion Site according to three possible transmission scenarios (high, moderate, and low). C_LIO_LIOur results suggest that a large-scale outbreak is very likely in this setting after a single infectious person enters the camp, with 0.5-91% of the population expected to be infected within the first three months and over 70-98% during the first year depending on the transmission scenario, should no effective interventions be put into place. C_LIO_LIHospitalization needs may exceed the existing hospitalization capacity of 340 beds after 55-136 days of introduction. C_LI O_LSTWhat do these findings mean?C_LSTO_LIA COVID-19 epidemic in a high population density refugee settlement may have profound consequences, requiring increases in healthcare capacity and infrastructure that exceed what is feasible in this setting. C_LIO_LIAs many of the approaches used to prevent and respond to COVID-19 in the most affected areas so far will not be practical in humanitarian settings, novel and untested strategies to protect the most vulnerable population groups should be considered, as well as innovative solutions to fill health workforce gaps. C_LI

The incubation period of 2019-nCoV from publicly reported confirmed cases: estimation and application

A novel human coronavirus (2019-nCoV) was identified in China in December, 2019. There is limited support for many of its key epidemiologic features, including the incubation period, which has important implications for surveillance and control activities. Here, we use data from public reports of 101 confirmed cases in 38 provinces, regions, and countries outside of Wuhan (Hubei province, China) with identifiable exposure windows and known dates of symptom onset to estimate the incubation period of 2019-nCoV. We estimate the median incubation period of 2019-nCoV to be 5.2 days (95% CI: 4.4, 6.0), and 97.5% of those who develop symptoms will do so within 10.5 days (95% CI: 7.3, 15.3) of infection. These estimates imply that, under conservative assumptions, 64 out of every 10,000 cases will develop symptoms after 14 days of active monitoring or quarantine. Whether this risk is acceptable depends on the underlying risk of infection and consequences of missed cases. The estimates presented here can be used to inform policy in multiple contexts based on these judgments.