Horten Centre for Patient Oriented Research and Knowledge Transfer, University Hospital Zurich, Zurich, Switzerland.
BMC Med Res Methodol. 2012 Jul 20;12:99. doi: 10.1186/1471-2288-12-99.
The development of risk prediction models is of increasing importance in medical research - their use in practice, however, is rare. Among other reasons this might be due to the fact that thorough validation is often lacking. This study focuses on two Bayesian approaches of how to validate a prediction rule for the diagnosis of pneumonia, and compares them with established validation methods.
Expert knowledge was used to derive a risk prediction model for pneumonia. Data on more than 600 patients presenting with cough and fever at a general practitioner's practice in Switzerland were collected in order to validate the expert model and to examine the predictive performance of it. Additionally, four modifications of the original model including shrinkage of the regression coefficients, and two Bayesian approaches with the expert model used as prior mean and different weights for the prior covariance matrix were fitted. We quantify the predictive performance of the different methods with respect to calibration and discrimination, using cross-validation.
The predictive performance of the unshrinked regression coefficients was poor when applied to the Swiss cohort. Shrinkage improved the results, but a Bayesian model formulation with unspecified weight of the informative prior lead to large AUC and small Brier score, naïve and after cross-validation. The advantage of this approach is the flexibility in case of a prior-data conflict.
Published risk prediction rules in clinical research need to be validated externally before they can be used in new settings. We propose to use a Bayesian model formulation with the original risk prediction rule as prior. The posterior means of the coefficients, given the validation data showed best predictive performance with respect to cross-validated calibration and discriminative ability.
风险预测模型的开发在医学研究中变得越来越重要——但它们在实践中的应用却很少。原因之一可能是缺乏彻底的验证。本研究关注了两种贝叶斯方法，用于验证肺炎诊断的预测规则，并将其与已建立的验证方法进行了比较。
利用专家知识推导出一个肺炎风险预测模型。为了验证专家模型并检验其预测性能，我们在瑞士的一家全科医生诊所收集了 600 多名出现咳嗽和发热症状的患者的数据。此外，我们还拟合了原始模型的四个修改版本，包括回归系数的收缩，以及两种贝叶斯方法，将专家模型作为先验均值，并为先验协方差矩阵赋予不同的权重。我们通过交叉验证来量化不同方法在校准和判别方面的预测性能。
未收缩的回归系数在应用于瑞士队列时预测性能不佳。收缩改善了结果，但在没有指定先验信息权重的贝叶斯模型公式下，会导致 AUC 较大，Brier 得分较小，无论是原始的还是经过交叉验证的。这种方法的优点是在存在先验-数据冲突的情况下具有灵活性。
临床研究中发表的风险预测规则需要在新环境中使用之前进行外部验证。我们建议使用贝叶斯模型公式，将原始风险预测规则作为先验。根据验证数据得出的系数后验均值在交叉验证校准和判别能力方面表现出最佳的预测性能。
