Luijken Kim, Song Jia, Groenwold Rolf H H
Department of Clinical Epidemiology, Leiden University Medical Center, Leiden, the Netherlands.
Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, the Netherlands.
Diagn Progn Res. 2022 Apr 7;6(1):7. doi: 10.1186/s41512-022-00121-1.
When a predictor variable is measured in similar ways at the derivation and validation setting of a prognostic prediction model, yet both differ from the intended use of the model in practice (i.e., "predictor measurement heterogeneity"), performance of the model at implementation needs to be inferred. This study proposed an analysis to quantify the impact of anticipated predictor measurement heterogeneity.
A simulation study was conducted to assess the impact of predictor measurement heterogeneity across validation and implementation setting in time-to-event outcome data. The use of the quantitative prediction error analysis was illustrated using an example of predicting the 6-year risk of developing type 2 diabetes with heterogeneity in measurement of the predictor body mass index.
In the simulation study, calibration-in-the-large of prediction models was poor and overall accuracy was reduced in all scenarios of predictor measurement heterogeneity. Model discrimination decreased with increasing random predictor measurement heterogeneity.
Heterogeneity of predictor measurements across settings of validation and implementation reduced predictive performance at implementation of prognostic models with a time-to-event outcome. When validating a prognostic model, the targeted clinical setting needs to be considered and analyses can be conducted to quantify the impact of anticipated predictor measurement heterogeneity on model performance at implementation.
当在预后预测模型的推导和验证设置中以相似方式测量预测变量，但两者在实际应用中均与模型的预期用途不同时（即“预测变量测量异质性”），需要推断模型在实施时的性能。本研究提出了一种分析方法，以量化预期的预测变量测量异质性的影响。
进行了一项模拟研究，以评估预测变量测量异质性在事件发生时间结局数据的验证和实施设置中的影响。通过一个预测体重指数测量存在异质性的情况下2型糖尿病6年发病风险的例子，说明了定量预测误差分析的应用。
在模拟研究中，在预测变量测量异质性的所有情况下，预测模型的大样本校准均较差，总体准确性降低。随着随机预测变量测量异质性的增加，模型辨别力下降。
验证和实施设置之间预测变量测量的异质性降低了具有事件发生时间结局的预后模型在实施时的预测性能。在验证预后模型时，需要考虑目标临床设置，并可进行分析以量化预期的预测变量测量异质性对模型实施性能的影响。
