Collaborative Hubs: Making the Most of Predictive Epidemic Modeling

ABSTRACT

[...]models can vary in terms of what data they use, what they assume about transmission, and what analytic approach they use to produce projections. Because of this, relying on one model is dangerous because there is no guarantee that one model's choices and assumptions will yield an accurate prediction. In many fields, there is a long tradition of combining multiple models to mitigate this limitation by providing a single prediction that summarizes the view of the participating models.7 There has been a growing interest in using ensemble methodologies in epidemiology, with notable efforts in forecasting, risk prediction, causal inference, and decision-making.8-12 COORDINATION, COLLABORATION, AND EVALUATION A modeling "hub" is a consortium of research groups organized around a particular scientific challenge. The US COVID-19 Forecast Hub ensemble (including many component models) has struggled to produce accurate forecasts of cases and hospitalizations during periods of rapidly changing epidemic dynamics, such as the US peak of the winter wave in early 2021 or the rapid increases associated with the Delta variant in summer 2021 or in winter 2021-2022.3 Likewise, although longer-term projections from the COVID-19 Scenario Modeling Hub projected a Delta-associated resurgence in the United States, the ensemble significantly underestimated its speed and size, even though there were no clear deviations from scenario assumptions.13 However, even when projections are wrong, the hubs play a role in enhancing the scientific rigor and integrity of epidemic modeling. [...]operationally, there is value in developing procedures that harness the insights of a diverse network of scientists while guarding against groupthink and overconfidence.12 As researchers, system developers, and public health officials who have been deeply involved in the real-time operation of modeling hubs duringthe COVID-19 pandemic and prior epidemics, we believe the hub approach is a vital path forward for predictive disease modeling efforts.