Department of Epidemiology & Preventive Medicine, Monash University, Melbourne, Victoria, Australia.
J Eval Clin Pract. 2013 Apr;19(2):358-62. doi: 10.1111/j.1365-2753.2012.01834.x. Epub 2012 Mar 12.
Development of chronic disease risk prediction models has become a growing area of research in recent years. The internal validity of such models is sometimes lower than estimated from the development sample. Overfitting or overoptimism of the developed model and/or differences between the samples are likely causes for this. For modelling of an uncommon outcome, bootstrapping for overoptimism is the preferred method for afterwards shrinking of regression coefficients and the model's discrimination and calibration for overoptimism. However, computer programs for different types of bootstrap validation are not readily available. We developed two SAS macro programs--one for the simple bootstrap that compares the discriminatory performance of the Cox proportional hazards model from the original sample in bootstrap samples; and another (which is more efficient), known as stepwise bootstrap validation, that makes the same comparison but from models developed by variable selection from bootstrap samples in the original sample. These are illustrated through an example from cardiovascular disease (CVD) risk prediction.
Two SAS macro programs for Cox proportional hazards model using Proc PHREG were developed for estimating overoptimism in Harrell's C and Somers' D statistics. The computer programs were applied to data on CVD incidence for a Framingham cohort that combined both the original and offspring exams. The risk factors considered were current smoking, diabetes, age, sex, systolic blood pressure, diastolic blood pressure, total cholesterol, high-density lipoprotein cholesterol, triglycerides and body mass index.
The degree of overoptimism in both Harrell's C and Somers' D statistics were low. Both these statistics were corrected for overoptimism by subtracting overoptimism from their observed values. Between the two bootstrap validation algorithms, the degree of overoptimism was estimated to be higher for stepwise bootstrap validation.
The programs are very useful for evaluating the 'overoptimism corrected' predictive performance of Cox proportional hazards model.
近年来，慢性病风险预测模型的开发已成为研究热点。此类模型的内部有效性有时低于从开发样本中估计的水平。模型开发过程中的过度拟合或过度拟合以及样本之间的差异可能是造成这种情况的原因。对于罕见结果的建模，自举法是减少回归系数和模型的判别力和校准过度拟合的首选方法。然而，不同类型的自举验证的计算机程序并不容易获得。我们开发了两个 SAS 宏程序 - 一个用于简单自举，比较原始样本中 Cox 比例风险模型在自举样本中的判别性能; 另一个（效率更高），称为逐步自举验证，从原始样本中自举样本中通过变量选择开发的模型进行相同的比较。通过心血管疾病（CVD）风险预测的一个例子来说明这一点。
使用 Proc PHREG 开发了两个用于估计 Harrell 的 C 和 Somers 的 D 统计量过度拟合的 Cox 比例风险模型的 SAS 宏程序。计算机程序应用于结合了原始和后代检查的弗雷明汉队列的 CVD 发病率数据。考虑的风险因素包括当前吸烟、糖尿病、年龄、性别、收缩压、舒张压、总胆固醇、高密度脂蛋白胆固醇、甘油三酯和体重指数。
Harrell 的 C 和 Somers 的 D 统计量的过度拟合程度较低。通过从观测值中减去过度拟合值，对这两个统计量进行了过度拟合的校正。在两种自举验证算法中，逐步自举验证的过度拟合程度估计更高。
这些程序对于评估 Cox 比例风险模型的“过度拟合校正”预测性能非常有用。
