Yogurtcu Osman N, Messan Marisabel Rodriguez, Gerkin Richard C, Belov Artur A, Yang Hong, Forshee Richard A, Chow Carson C
Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research, US FDA, 10903 New Hampshire Ave, Silver Spring, 20993, Maryland, USA.
School of Life Sciences, Arizona State University, Tempe, 85287, Arizona, USA.
medRxiv. 2021 Feb 8:2021.02.06.21251276. doi: 10.1101/2021.02.06.21251276.
Quantifying how accurate epidemiological models of COVID-19 forecast the number of future cases and deaths can help frame how to incorporate mathematical models to inform public health decisions. Here we analyze and score the predictive ability of publicly available COVID-19 epidemiological models on the COVID-19 Forecast Hub. Our score uses the posted forecast cumulative distributions to compute the log-likelihood for held-out COVID-19 positive cases and deaths. Scores are updated continuously as new data become available, and model performance is tracked over time. We use model scores to construct ensemble models based on past performance. Our publicly available quantitative framework may aid in improving modeling frameworks, and assist policy makers in selecting modeling paradigms to balance the delicate trade-offs between the economy and public health.
量化新冠疫情流行病学模型预测未来病例数和死亡数的准确程度，有助于确定如何运用数学模型为公共卫生决策提供信息。在此，我们对新冠疫情预测中心公开的新冠疫情流行病学模型的预测能力进行分析和评分。我们的评分利用公布的预测累积分布，来计算预留的新冠确诊病例数和死亡数的对数似然值。随着新数据的出现，评分会持续更新，并对模型性能进行长期跟踪。我们根据过去的表现，利用模型评分构建集成模型。我们公开的定量框架可能有助于改进建模框架，并协助政策制定者选择建模范式，以平衡经济与公共卫生之间的微妙权衡。
