Division of Health Policy and Management, University of Minnesota School of Public Health, Minneapolis, MN.
Division of Epidemiology and Community Health, University of Minnesota School of Public Health, Minneapolis, MN.
Med Decis Making. 2018 Oct;38(7):810-821. doi: 10.1177/0272989X18792283.
Calibration is the process of estimating parameters of a mathematical model by matching model outputs to calibration targets. In the presence of nonidentifiability, multiple parameter sets solve the calibration problem, which may have important implications for decision making. We evaluate the implications of nonidentifiability on the optimal strategy and provide methods to check for nonidentifiability.
We illustrate nonidentifiability by calibrating a 3-state Markov model of cancer relative survival (RS). We performed 2 different calibration exercises: 1) only including RS as a calibration target and 2) adding the ratio between the 2 nondeath states over time as an additional target. We used the Nelder-Mead (NM) algorithm to identify parameter sets that best matched the calibration targets. We used collinearity and likelihood profile analyses to check for nonidentifiability. We then estimated the benefit of a hypothetical treatment in terms of life expectancy gains using different, but equally good-fitting, parameter sets. We also applied collinearity analysis to a realistic model of the natural history of colorectal cancer.
When only RS is used as the calibration target, 2 different parameter sets yield similar maximum likelihood values. The high collinearity index and the bimodal likelihood profile on both parameters demonstrated the presence of nonidentifiability. These different, equally good-fitting parameter sets produce different estimates of the treatment effectiveness (0.67 v. 0.31 years), which could influence the optimal decision. By incorporating the additional target, the model becomes identifiable with a collinearity index of 3.5 and a unimodal likelihood profile.
In the presence of nonidentifiability, equally likely parameter estimates might yield different conclusions. Checking for the existence of nonidentifiability and its implications should be incorporated into standard model calibration procedures.
校准是通过将模型输出与校准目标匹配来估计数学模型参数的过程。在存在不可识别性的情况下,多个参数集可以解决校准问题,这可能对决策有重要影响。我们评估了不可识别性对最优策略的影响,并提供了检查不可识别性的方法。
我们通过校准癌症相对生存率 (RS) 的 3 状态马尔可夫模型来说明不可识别性。我们进行了 2 项不同的校准练习:1)仅将 RS 作为校准目标,2)随时间添加 2 个非死亡状态之间的比率作为附加目标。我们使用 Nelder-Mead(NM)算法来确定与校准目标最佳匹配的参数集。我们使用共线性和似然轮廓分析来检查不可识别性。然后,我们使用不同的、但拟合度相同的参数集来估计假设治疗在预期寿命收益方面的益处。我们还对结直肠癌自然史的实际模型进行了共线性分析。
当仅将 RS 用作校准目标时,2 个不同的参数集产生相似的最大似然值。高共线性指数和两个参数的双峰似然轮廓表明存在不可识别性。这些不同的、拟合度相同的参数集产生了不同的治疗效果估计(0.67 年与 0.31 年),这可能会影响最优决策。通过纳入附加目标,模型变得可识别,共线性指数为 3.5,似然轮廓为单峰。
在存在不可识别性的情况下,同样可能的参数估计可能会得出不同的结论。应将检查不可识别性的存在及其影响纳入标准模型校准程序中。