Challenges of COVID-19 Case Forecasting in the US, 2020-2021 (preprint)

During the COVID-19 pandemic, forecasting COVID-19 trends to support planning and response was a priority for scientists and decision makers alike. In the United States, COVID-19 forecasting was coordinated by a large group of universities, companies, and government entities led by the Centers for Disease Control and Prevention and the US COVID-19 Forecast Hub (https://covid19forecasthub.org). We evaluated approximately 9.7 million forecasts of weekly state-level COVID-19 cases for predictions 1-4 weeks into the future submitted by 24 teams from August 2020 to December 2021. We assessed coverage of central prediction intervals and weighted interval scores (WIS), adjusting for missing forecasts relative to a baseline forecast, and used a Gaussian generalized estimating equation (GEE) model to evaluate differences in skill across epidemic phases that were defined by the effective reproduction number. Overall, we found high variation in skill across individual models, with ensemble-based forecasts outperforming other approaches. Forecast skill relative to the baseline was generally higher for larger jurisdictions (e.g., states compared to counties). Over time, forecasts generally performed worst in periods of rapid changes in reported cases (either in increasing or decreasing epidemic phases) with 95% prediction interval coverage dropping below 50% during the growth phases of the winter 2020, Delta, and Omicron waves. Ideally, case forecasts could serve as a leading indicator of changes in transmission dynamics. However, while most COVID-19 case forecasts outperformed a naive baseline model, even the most accurate case forecasts were unreliable in key phases. Further research could improve forecasts of leading indicators, like COVID-19 cases, by leveraging additional real-time data, addressing performance across phases, improving the characterization of forecast confidence, and ensuring that forecasts were coherent across spatial scales. In the meantime, it is critical for forecast users to appreciate current limitations and use a broad set of indicators to inform pandemic-related decision making. Author SummaryAs SARS-CoV-2 began to spread throughout the world in early 2020, modelers played a critical role in predicting how the epidemic could take shape. Short-term forecasts of epidemic outcomes (for example, infections, cases, hospitalizations, or deaths) provided useful information to support pandemic planning, resource allocation, and intervention. Yet, infectious disease forecasting is still a nascent science, and the reliability of different types of forecasts is unclear. We retrospectively evaluated COVID-19 case forecasts, which were often unreliable. For example, forecasts did not anticipate the speed of increase in cases in early winter 2020. This analysis provides insights on specific problems that could be addressed in future research to improve forecasts and their use. Identifying the strengths and weaknesses of forecasts is critical to improving forecasting for current and future public health responses.

COFFEE: COVID-19 Forecasts using Fast Evaluations and Estimation (preprint)

This document details the methodology of the Los Alamos National Laboratory COVID-19 forecasting model, COFFEE (COVID-19 Forecasts using Fast Evaluations and Estimation).

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US (preprint)

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. In 2020, the COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized hundreds of thousands of specific predictions from more than 50 different academic, industry, and independent research groups. This manuscript systematically evaluates 23 models that regularly submitted forecasts of reported weekly incident COVID-19 mortality counts in the US at the state and national level. One of these models was a multi-model ensemble that combined all available forecasts each week. The performance of individual models showed high variability across time, geospatial units, and forecast horizons. Half of the models evaluated showed better accuracy than a naive baseline model. In combining the forecasts from all teams, the ensemble showed the best overall probabilistic accuracy of any model. Forecast accuracy degraded as models made predictions farther into the future, with probabilistic accuracy at a 20-week horizon more than 5 times worse than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. f

Short-term forecasting of COVID-19 in Germany and Poland during the second wave - a preregistered study (preprint)

We report insights from ten weeks of collaborative COVID-19 forecasting for Germany and Poland (12 October - 19 December 2020). The study period covers the onset of the second wave in both countries, with tightening non-pharmaceutical interventions (NPIs) and subsequently a decay (Poland) or plateau and renewed increase (Germany) in reported cases. Thirteen independent teams provided probabilistic real-time forecasts of COVID-19 cases and deaths. These were reported for lead times of one to four weeks, with evaluation focused on one- and two-week horizons, which are less affected by changing NPIs. Heterogeneity between forecasts was considerable both in terms of point predictions and forecast spread. Ensemble forecasts showed good relative performance, in particular in terms of coverage, but did not clearly dominate single-model predictions. The study was preregistered and will be followed up in future phases of the pandemic.

"Thought I'd Share First" and Other Conspiracy Theory Tweets from the COVID-19 Infodemic: Exploratory Study (preprint)

Background: The COVID-19 outbreak has left many people isolated within their homes; these people are turning to social media for news and social connection, which leaves them vulnerable to believing and sharing misinformation. Health-related misinformation threatens adherence to public health messaging, and monitoring its spread on social media is critical to understanding the evolution of ideas that have potentially negative public health impacts. Results: Analysis using model-labeled data was beneficial for increasing the proportion of data matching misinformation indicators. Random forest classifier metrics varied across the four conspiracy theories considered (F1 scores between 0.347 and 0.857); this performance increased as the given conspiracy theory was more narrowly defined. We showed that misinformation tweets demonstrate more negative sentiment when compared to nonmisinformation tweets and that theories evolve over time, incorporating details from unrelated conspiracy theories as well as real-world events. Conclusions: Although we focus here on health-related misinformation, this combination of approaches is not specific to public health and is valuable for characterizing misinformation in general, which is an important first step in creating targeted messaging to counteract its spread. Initial messaging should aim to preempt generalized misinformation before it becomes widespread, while later messaging will need to target evolving conspiracy theories and the new facets of each as they become incorporated.

COVID-19 reopening strategies at the county level in the face of uncertainty: Multiple Models for Outbreak Decision Support (preprint)

Policymakers make decisions about COVID-19 management in the face of considerable uncertainty. We convened multiple modeling teams to evaluate reopening strategies for a mid-sized county in the United States, in a novel process designed to fully express scientific uncertainty while reducing linguistic uncertainty and cognitive biases. For the scenarios considered, the consensus from 17 distinct models was that a second outbreak will occur within 6 months of reopening, unless schools and non-essential workplaces remain closed. Up to half the population could be infected with full workplace reopening; non-essential business closures reduced median cumulative infections by 82%. Intermediate reopening interventions identified no win-win situations; there was a trade-off between public health outcomes and duration of workplace closures. Aggregate results captured twice the uncertainty of individual models, providing a more complete expression of risk for decision-making purposes.

The N-terminal domain of spike glycoprotein mediates SARS-CoV-2 infection by associating with L-SIGN and DC-SIGN (preprint)

The widespread occurrence of SARS-CoV-2 has had a profound effect on society and a vaccine is currently being developed. Angiotensin-converting enzyme 2 (ACE2) is the primary host cell receptor that interacts with the receptor-binding domain (RBD) of the SARS-CoV-2 spike protein. Although pneumonia is the main symptom in severe cases of SARS-CoV-2 infection, the expression levels of ACE2 in the lung is low, suggesting the presence of another receptor for the spike protein. In order to identify the additional receptors for the spike protein, we screened a receptor for the SARS-CoV-2 spike protein from the lung cDNA library. We cloned L-SIGN as a specific receptor for the N-terminal domain (NTD) of the SARS-CoV-2 spike protein. The RBD of the spike protein did not bind to L-SIGN. In addition, not only L-SIGN but also DC-SIGN, a closely related C-type lectin receptor to L-SIGN, bound to the NTD of the SARS-CoV-2 spike protein. Importantly, cells expressing L-SIGN and DC-SIGN were both infected by SARS-CoV-2. Furthermore, L-SIGN and DC-SIGN induced membrane fusion by associating with the SARS-CoV-2 spike protein. Serum antibodies from infected patients and a patient-derived monoclonal antibody against NTD inhibited SARS-CoV-2 infection of L-SIGN or DC-SIGN expressing cells. Our results highlight the important role of NTD in SARS-CoV-2 dissemination through L-SIGN and DC-SIGN and the significance of having anti-NTD neutralizing antibodies in antibody-based therapeutics.

Comparison of SARS-CoV-2 infection in two non-human primate species: rhesus and cynomolgus macaques (preprint)

SARS-CoV-2 is a coronavirus that sparked the current COVID-19 pandemic. To stop the shattering effect of COVID-19, effective and safe vaccines, and antiviral therapies are urgently needed. To facilitate the preclinical evaluation of intervention approaches, relevant animal models need to be developed and validated. Rhesus macaques (Macaca mulatta) and cynomolgus macaques (Macaca fascicularis) are widely used in biomedical research and serve as models for SARS-CoV-2 infection. However, differences in study design make it difficult to compare and understand potential species-related differences. Here, we directly compared the course of SARS-CoV-2 infection in the two genetically closely-related macaque species. After inoculation with a low passage SARS-CoV-2 isolate, clinical, virological, and immunological characteristics were monitored. Both species showed slightly elevated body temperatures in the first days after exposure while a decrease in physical activity was only observed in the rhesus macaques and not in cynomolgus macaques. The virus was quantified in tracheal, nasal, and anal swabs, and in blood samples by qRT-PCR, and showed high similarity between the two species. Immunoglobulins were detected by various enzyme-linked immunosorbent assays (ELISAs) and showed seroconversion in all animals by day 10 post-infection. The cytokine responses were highly comparable between species and computed tomography (CT) imaging revealed pulmonary lesions in all animals. Consequently, we concluded that both rhesus and cynomolgus macaques represent valid models for evaluation of COVID-19 vaccine and antiviral candidates in a preclinical setting.

A Workflow of Integrated Resources to Catalyze Network Pharmacology Driven COVID-19 Research (preprint)

Motivation: In the event of an outbreak due to an emerging pathogen, time is of the essence to contain or to mitigate the spread of the disease. Drug repositioning is one of the strategies that has the potential to deliver therapeutics relatively quickly. The SARS-CoV-2 pandemic has shown that integrating critical data resources to drive drug-repositioning studies, involving host-host, host-pathogen and drug-target interactions, remains a time-consuming effort that translates to a delay in the development and delivery of a life-saving therapy. Results: Here, we describe a workflow we designed for a semi-automated integration of rapidly emerging datasets that can be generally adopted in a broad network pharmacology research setting. The workflow was used to construct a COVID-19 focused multimodal network that integrates 487 host-pathogen, 74,805 host-host protein and 1,265 drug-target interactions. The resultant Neo4j graph database named "Neo4COVID19" is accessible via a web interface and via API calls based on the Bolt protocol. We believe that our Neo4COVID19 database will be a valuable asset to the research community and will catalyze the discovery of therapeutics to fight COVID-19. Availability: https://neo4covid19.ncats.io . Keywords: SARS-CoV-2, COVID-19, network pharmacology, graph database, Neo4j, data integration, drug repositioning

Using an Agent-Based Model to Assess K-12 School Re-openings Under Different COVID-19 Spread Scenarios - United States, School Year 2020/21 (preprint)

School-age children play a key role in the spread of airborne viruses like influenza due to the prolonged and close contacts they have in school settings. As a result, school closures and other non-pharmaceutical interventions were recommended as the first line of defense in response to the novel coronavirus pandemic (COVID-19). Assessing school reopening scenarios is a priority for states, administrators, parents, and children in order to balance educational disparities and negative population impacts of COVID-19. To address this challenge, we used an agent-based model that simulates communities across the United States including daycares, primary, and secondary schools to quantify the relative health outcomes of reopening schools. We explored different reopening scenarios including remote learning, in-person school, and several hybrid options that stratify the student population into cohorts (i.e., split cohort) in order to reduce exposure and disease spread. In addition, we assessed the combined impact of reduced in-person attendance in workplaces (e.g., through differing degrees of reliance on telework and/or temporary workplace closings) and school reopening scenarios to quantify the potential impact of additional transmission pathways contributing to COVID-19 spread. Scenarios where split cohorts of students return to school in non-overlapping formats resulted in significant decreases in the clinical attack rate (i.e., the percentage of symptomatic individuals), potentially by as much as 75% . These split cohort scenarios have impacts which are only modestly lesser than the most impactful 100% distance learning scenario. Split cohort scenarios can also significantly avert the number of cases--approximately 60M and 28M--depending on the scenario, at the national scale over the simulated eight-month period. We found the results of our simulations to be highly dependent on the number of workplaces assumed to be open for in-person business, as well as the initial level of COVID-19 incidence within the simulated community. Our results show that reducing the number of students attending school leads to better health outcomes, and the split cohort option enables part-time in-classroom education while substantially reducing risk. The results of this study can support decisions regarding optimal school reopening strategies that at the population level balance education and the negative health outcomes of COVID-19.