Demler Olga V, Pencina Michael J, Cook Nancy R, D'Agostino Ralph B
Division of Preventive Medicine, Brigham and Women's Hospital, 900 Commonwealth Avenue, Boston, MA, 02115, U.S.A.
Department of Biostatistics and Bioinformatics, Duke University, Durham, NC, 27708, U.S.A.
Stat Med. 2017 Sep 20;36(21):3334-3360. doi: 10.1002/sim.7333. Epub 2017 Jun 19.
The change in area under the curve (∆AUC), the integrated discrimination improvement (IDI), and net reclassification index (NRI) are commonly used measures of risk prediction model performance. Some authors have reported good validity of associated methods of estimating their standard errors (SE) and construction of confidence intervals, whereas others have questioned their performance. To address these issues, we unite the ∆AUC, IDI, and three versions of the NRI under the umbrella of the U-statistics family. We rigorously show that the asymptotic behavior of ∆AUC, NRIs, and IDI fits the asymptotic distribution theory developed for U-statistics. We prove that the ∆AUC, NRIs, and IDI are asymptotically normal, unless they compare nested models under the null hypothesis. In the latter case, asymptotic normality and existing SE estimates cannot be applied to ∆AUC, NRIs, or IDI. In the former case, SE formulas proposed in the literature are equivalent to SE formulas obtained from U-statistics theory if we ignore adjustment for estimated parameters. We use Sukhatme-Randles-deWet condition to determine when adjustment for estimated parameters is necessary. We show that adjustment is not necessary for SEs of the ∆AUC and two versions of the NRI when added predictor variables are significant and normally distributed. The SEs of the IDI and three-category NRI should always be adjusted for estimated parameters. These results allow us to define when existing formulas for SE estimates can be used and when resampling methods such as the bootstrap should be used instead when comparing nested models. We also use the U-statistic theory to develop a new SE estimate of ∆AUC. Copyright © 2017 John Wiley & Sons, Ltd.
曲线下面积变化（∆AUC）、综合判别改善（IDI）和净重新分类指数（NRI）是常用的风险预测模型性能度量指标。一些作者报告了估计其标准误差（SE）和构建置信区间的相关方法具有良好的有效性，而另一些作者则对其性能提出了质疑。为了解决这些问题，我们将∆AUC、IDI和三个版本的NRI统一在U统计量族的框架下。我们严格证明了∆AUC、NRI和IDI的渐近行为符合为U统计量发展的渐近分布理论。我们证明了∆AUC、NRI和IDI渐近正态，除非它们在原假设下比较嵌套模型。在后一种情况下，渐近正态性和现有的SE估计不能应用于∆AUC、NRI或IDI。在前一种情况下，如果我们忽略对估计参数的调整，文献中提出的SE公式与从U统计量理论获得的SE公式等价。我们使用Sukhatme-Randles-deWet条件来确定何时需要对估计参数进行调整。我们表明，当添加的预测变量显著且呈正态分布时，∆AUC和两个版本的NRI的SE不需要调整。IDI和三类NRI的SE应始终对估计参数进行调整。这些结果使我们能够确定何时可以使用现有的SE估计公式，以及在比较嵌套模型时何时应使用诸如自助法等重采样方法。我们还使用U统计量理论开发了一种新的∆AUC的SE估计。版权所有© 2017约翰威立父子有限公司。
