Corro Ramos Isaac, van Voorn George A K, Vemer Pepijn, Feenstra Talitha L, Al Maiwenn J
Institute for Medical Technology Assessment, Erasmus University Rotterdam, Rotterdam, The Netherlands.
Biometris, Wageningen University and Research, Wageningen, The Netherlands.
Value Health. 2017 Sep;20(8):1041-1047. doi: 10.1016/j.jval.2017.04.016. Epub 2017 May 29.
The validation of health economic (HE) model outcomes against empirical data is of key importance. Although statistical testing seems applicable, guidelines for the validation of HE models lack guidance on statistical validation, and actual validation efforts often present subjective judgment of graphs and point estimates.
To discuss the applicability of existing validation techniques and to present a new method for quantifying the degrees of validity statistically, which is useful for decision makers.
A new Bayesian method is proposed to determine how well HE model outcomes compare with empirical data. Validity is based on a pre-established accuracy interval in which the model outcomes should fall. The method uses the outcomes of a probabilistic sensitivity analysis and results in a posterior distribution around the probability that HE model outcomes can be regarded as valid.
We use a published diabetes model (Modelling Integrated Care for Diabetes based on Observational data) to validate the outcome "number of patients who are on dialysis or with end-stage renal disease." Results indicate that a high probability of a valid outcome is associated with relatively wide accuracy intervals. In particular, 25% deviation from the observed outcome implied approximately 60% expected validity.
Current practice in HE model validation can be improved by using an alternative method based on assessing whether the model outcomes fit to empirical data at a predefined level of accuracy. This method has the advantage of assessing both model bias and parameter uncertainty and resulting in a quantitative measure of the degree of validity that penalizes models predicting the mean of an outcome correctly but with overly wide credible intervals.
将卫生经济（HE）模型结果与实证数据进行验证至关重要。尽管统计检验似乎适用，但HE模型验证指南缺乏关于统计验证的指导，实际验证工作往往是对图表和点估计进行主观判断。
讨论现有验证技术的适用性，并提出一种用于统计量化有效性程度的新方法，这对决策者很有用。
提出一种新的贝叶斯方法，以确定HE模型结果与实证数据的匹配程度。有效性基于预先确定的准确性区间，模型结果应落在该区间内。该方法使用概率敏感性分析的结果，并得出围绕HE模型结果可被视为有效的概率的后验分布。
我们使用一个已发表的糖尿病模型（基于观察数据的糖尿病综合护理建模）来验证“接受透析或患有终末期肾病的患者数量”这一结果。结果表明，有效结果的高概率与相对较宽的准确性区间相关。特别是，与观察结果有25%的偏差意味着预期有效性约为60%。
通过使用一种基于评估模型结果在预定义准确性水平上是否符合实证数据的替代方法，可以改进HE模型验证的当前实践。该方法具有评估模型偏差和参数不确定性的优点，并能得出有效性程度的定量度量，对正确预测结果均值但可信区间过宽的模型进行惩罚。
