South African DSI-NRF Centre of Excellence in Epidemiological Modelling and Analysis (SACEMA), Stellenbosch University, Stellenbosch, South Africa.
Department of Biology, Department of Mathematics and Statistics, Institute for Infectious Disease Research, McMaster University, Hamilton, Ontario, Canada.
PLoS Comput Biol. 2020 May 11;16(5):e1007893. doi: 10.1371/journal.pcbi.1007893. eCollection 2020 May.
Individual-based models (IBMs) informing public health policy should be calibrated to data and provide estimates of uncertainty. Two main components of model-calibration methods are the parameter-search strategy and the goodness-of-fit (GOF) measure; many options exist for each of these. This review provides an overview of calibration methods used in IBMs modelling infectious disease spread. We identified articles on PubMed employing simulation-based methods to calibrate IBMs informing public health policy in HIV, tuberculosis, and malaria epidemiology published between 1 January 2013 and 31 December 2018. Articles were included if models stored individual-specific information, and calibration involved comparing model output to population-level targets. We extracted information on parameter-search strategies, GOF measures, and model validation. The PubMed search identified 653 candidate articles, of which 84 met the review criteria. Of the included articles, 40 (48%) combined a quantitative GOF measure with an algorithmic parameter-search strategy-either an optimisation algorithm (14/40) or a sampling algorithm (26/40). These 40 articles varied widely in their choices of parameter-search strategies and GOF measures. For the remaining 44 (52%) articles, the parameter-search strategy could either not be identified (32/44) or was described as an informal, non-reproducible method (12/44). Of these 44 articles, the majority (25/44) were unclear about the GOF measure used; of the rest, only five quantitatively evaluated GOF. Only a minority of the included articles, 14 (17%) provided a rationale for their choice of model-calibration method. Model validation was reported in 31 (37%) articles. Reporting on calibration methods is far from optimal in epidemiological modelling studies of HIV, malaria and TB transmission dynamics. The adoption of better documented, algorithmic calibration methods could improve both reproducibility and the quality of inference in model-based epidemiology. There is a need for research comparing the performance of calibration methods to inform decisions about the parameter-search strategies and GOF measures.
个体为基础的模型（IBMs）为公共卫生政策提供信息时，应根据数据进行校准，并提供不确定性的估计。模型校准方法的两个主要组成部分是参数搜索策略和拟合优度（GOF）度量；对于每个组成部分，都有许多选项。本综述提供了在 HIV、结核病和疟疾流行病学的 IBM 传染病传播建模中使用的校准方法概述。我们在 PubMed 上确定了 2013 年 1 月 1 日至 2018 年 12 月 31 日期间发表的使用基于模拟的方法校准告知公共卫生政策的 IBM 的文章。如果模型存储个体特定信息且校准涉及将模型输出与人群水平目标进行比较，则将文章纳入。我们提取了有关参数搜索策略、GOF 度量和模型验证的信息。PubMed 搜索确定了 653 个候选文章，其中 84 个符合审查标准。在纳入的文章中，40 篇（48%）将定量 GOF 度量与算法参数搜索策略相结合-优化算法（14/40）或抽样算法（26/40）。这些 40 篇文章在参数搜索策略和 GOF 度量的选择上差异很大。对于其余 44 篇（52%）文章，无法确定参数搜索策略（32/44）或将其描述为非正式、不可重复的方法（12/44）。在这 44 篇文章中，大多数（25/44）未说明使用的 GOF 度量；其余文章中，只有 5 篇定量评估了 GOF。在纳入的文章中，只有少数（14/17%）为其选择的模型校准方法提供了理由。31 篇（37%）文章报告了模型验证。在 HIV、疟疾和结核病传播动力学的流行病学建模研究中，校准方法的报告远非最佳。采用记录更好、算法化的校准方法可以提高基于模型的流行病学中的可重复性和推断质量。需要进行研究比较校准方法的性能，以便就参数搜索策略和 GOF 度量做出决策。
