RESUMEN
Academic researchers, government agencies, industry groups, and individuals have produced forecasts at an unprecedented scale during the COVID-19 pandemic. To leverage these forecasts, the United States Centers for Disease Control and Prevention (CDC) partnered with an academic research lab at the University of Massachusetts Amherst to create the US COVID-19 Forecast Hub. Launched in April 2020, the Forecast Hub is a dataset with point and probabilistic forecasts of incident hospitalizations, incident cases, incident deaths, and cumulative deaths due to COVID-19 at national, state, and county levels in the United States. Included forecasts represent a variety of modeling approaches, data sources, and assumptions regarding the spread of COVID-19. The goal of this dataset is to establish a standardized and comparable set of short-term forecasts from modeling teams. These data can be used to develop ensemble models, communicate forecasts to the public, create visualizations, compare models, and inform policies regarding COVID-19 mitigation. These open-source data are available via download from GitHub, through an online API, and through R packages.
RESUMEN
The rapid spread of SARS-CoV-2 is largely driven by pre-symptomatic or mildly symptomatic individuals who transmit the virus. Serological tests to identify antibodies against SARS-CoV-2 are an important tool to characterize subclinical infection exposure, which is critical in determining transmission trajectories and consequent population immunity. During the summer of 2020, a mail-based serological survey with self-collected dried blood spot (DBS) samples was implemented among university affiliates and their household members in Massachusetts, USA. Described here are some of the challenges faced and novel procedures used during the implementation of this study to assess the prevalence of SARS-CoV-2 antibodies amid the global pandemic. Important challenges included remote and contact-minimized participant recruitment, limited availability of commodities and laboratory capacity, a potentially biased sample population, and policy changes impacting the distribution of clinical results to study participants. Methods used to surmount these challenges and lessons learned are presented to inform similar studies. Key lessons relate to the acceptability and feasibility of DBS sampling, supply requirements, the logistics of packing and shipping packages, data linkages to enrolled household members, and the utility of having an on-call nurse available for participant concerns during sample collection. Future studies might consider additional recruitment techniques such as conducting studies during academic semesters when recruiting in a university setting, partnerships with supply and shipping specialists, and using a stratified sampling approach to minimize potential biases in recruitment. This study design highlights the feasibility and acceptability of self-collected bio-samples and has broad applicability for other serological surveys for a range of pathogens.
RESUMEN
BackgroundThe SARS-CoV-2 pandemic is an unprecedented global health crisis. The state of Massachusetts was especially impacted during the initial stages; however, the extent of asymptomatic transmission remains poorly understood due to limited asymptomatic testing in the "first wave." To address this gap, a geographically representative and contact-free seroprevalence survey was conducted in July-August 2020, to estimate prior undetected SARS-CoV-2 infections. MethodsStudents, faculty, librarians and staff members at the University of Massachusetts, Amherst without a previous COVID-19 diagnosis were invited to participate in this study along with one member of their household in June 2020. Two separate sampling frames were generated from administrative lists: all undergraduates and their household members (primary sampling group) were randomly selected with probability proportional to population size. All staff, faculty, graduate students and librarians (secondary sampling group) were selected as a simple random sample. After informed consent and a socio-behavioral survey, participants were mailed test kits and asked to return self-collected dried blood spot (DBS) samples. Samples were analyzed via ELISA for anti-SARS-CoV-2 IgG antibodies, and then IgM antibodies if IgG-positive. Seroprevalence estimates were adjusted for survey non-response. Binomial models were used to assess factors associated with seropositivity in both sample groups separately. ResultsApproximately 27,000 persons were invited via email to assess eligibility. Of the 1,001 individuals invited to participate in the study, 762 (76%) returned blood samples for analysis. In the primary sampling group 548 returned samples, of which 230 enrolled a household member. Within the secondary sampling group of 214 individuals, 79 enrolled a household member. In the primary sample group, 36 (4.6%) had IgG antibodies detected for an estimated weighed prevalence for this population of 5.3% (95% CI: 3.5 to 8.0). In the secondary sampling group, 10 (3.4%) of 292 individuals had IgG antibodies detected for an estimated adjusted prevalence of 4.0% (95% CI: 2.2 to 7.4). No samples were IgM positive. No association was found in either sample group between seropositivity and self-reported work duties or customer-facing hours. In the primary sampling group, self-reported febrile illness since Feb 2020, male sex, and minority race (Black or American Indian/Alaskan Native) were associated with seropositivity. No factors except geographic regions within the state were associated with evidence of prior SARS-CoV-2 infection in the secondary sampling group. InterpretationThis study provides insight into the seroprevalence of university-related populations and their household members across the state of Massachusetts during the summer of 2020 of the pandemic and helps to fill a critical gap in estimating the levels of sub-clinical and asymptomatic infection. Estimates like these can be used to calibrate models that estimate levels of population immunity over time to inform public health interventions and policy. FundingUMass Faculty Fund (A Lover); UMass Faculty Discretionary Funds (N Reich); UMass Institute for Applied Life Science "Midigrant" (#169076; A. Lover); and D. Alfandari was supported by grants from the USPHS (R24OD021485).
RESUMEN
Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. Starting in April 2020, the US COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized tens of millions of specific predictions from more than 90 different academic, industry, and independent research groups. A multi-model ensemble forecast that combined predictions from dozens of different research groups every week provided the most consistently accurate probabilistic forecasts of incident deaths due to COVID-19 at the state and national level from April 2020 through October 2021. The performance of 27 individual models that submitted complete forecasts of COVID-19 deaths consistently throughout this year showed high variability in forecast skill across time, geospatial units, and forecast horizons. Two-thirds of the models evaluated showed better accuracy than a naive baseline model. Forecast accuracy degraded as models made predictions further into the future, with probabilistic error at a 20-week horizon 3-5 times larger than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. Significance StatementThis paper compares the probabilistic accuracy of short-term forecasts of reported deaths due to COVID-19 during the first year and a half of the pandemic in the US. Results show high variation in accuracy between and within stand-alone models, and more consistent accuracy from an ensemble model that combined forecasts from all eligible models. This demonstrates that an ensemble model provided a reliable and comparatively accurate means of forecasting deaths during the COVID-19 pandemic that exceeded the performance of all of the models that contributed to it. This work strengthens the evidence base for synthesizing multiple models to support public health action.
RESUMEN
BackgroundThe COVID-19 pandemic has driven demand for forecasts to guide policy and planning. Previous research has suggested that combining forecasts from multiple models into a single "ensemble" forecast can increase the robustness of forecasts. Here we evaluate the real-time application of an open, collaborative ensemble to forecast deaths attributable to COVID-19 in the U.S. MethodsBeginning on April 13, 2020, we collected and combined one- to four-week ahead forecasts of cumulative deaths for U.S. jurisdictions in standardized, probabilistic formats to generate real-time, publicly available ensemble forecasts. We evaluated the point prediction accuracy and calibration of these forecasts compared to reported deaths. ResultsAnalysis of 2,512 ensemble forecasts made April 27 to July 20 with outcomes observed in the weeks ending May 23 through July 25, 2020 revealed precise short-term forecasts, with accuracy deteriorating at longer prediction horizons of up to four weeks. At all prediction horizons, the prediction intervals were well calibrated with 92-96% of observations falling within the rounded 95% prediction intervals. ConclusionsThis analysis demonstrates that real-time, publicly available ensemble forecasts issued in April-July 2020 provided robust short-term predictions of reported COVID-19 deaths in the United States. With the ongoing need for forecasts of impacts and resource needs for the COVID-19 response, the results underscore the importance of combining multiple probabilistic models and assessing forecast skill at different prediction horizons. Careful development, assessment, and communication of ensemble forecasts can provide reliable insight to public health decision makers.
