Division of Infectious Diseases, Wadsworth Center, New York State Department of Health, Albany, NY, United States of America.
Department of Atmospheric and Environmental Sciences, University at Albany, SUNY, Albany, NY, United States of America.
PLoS One. 2023 May 5;18(5):e0285215. doi: 10.1371/journal.pone.0285215. eCollection 2023.
Null models provide a critical baseline for the evaluation of predictive disease models. Many studies consider only the grand mean null model (i.e. R2) when evaluating the predictive ability of a model, which is insufficient to convey the predictive power of a model. We evaluated ten null models for human cases of West Nile virus (WNV), a zoonotic mosquito-borne disease introduced to the United States in 1999. The Negative Binomial, Historical (i.e. using previous cases to predict future cases) and Always Absent null models were the strongest overall, and the majority of null models significantly outperformed the grand mean. The length of the training timeseries increased the performance of most null models in US counties where WNV cases were frequent, but improvements were similar for most null models, so relative scores remained unchanged. We argue that a combination of null models is needed to evaluate the forecasting performance of predictive models for infectious diseases and the grand mean is the lowest bar.
零模型为评估预测疾病模型提供了重要的基准。许多研究在评估模型的预测能力时仅考虑总体均值零模型（即 R2），这不足以传达模型的预测能力。我们评估了十种针对人类西尼罗河病毒（WNV）病例的零模型，WNV 是一种由蚊子传播的动物源性疾病，于 1999 年引入美国。负二项式、历史（即使用以前的病例来预测未来的病例）和总是不存在零模型总体上最强，大多数零模型的表现明显优于总体均值。在 WNV 病例频繁的美国县，训练时间序列的长度增加了大多数零模型的性能，但大多数零模型的改进相似，因此相对分数保持不变。我们认为，需要结合使用多种零模型来评估传染病预测模型的预测性能，而总体均值是最低标准。
