Zhang Zhongheng, Rousson Valentin, Lee Wen-Chung, Ferdynus Cyril, Chen Mingyu, Qian Xin, Guo Yizhan
Department of Emergency Medicine, Sir Run Run Shaw Hospital, Zhejiang University School of Medicine, Hangzhou 310016, China.
Division of Biostatistics, Institute for Social and Preventive Medicine, University Hospital Lausanne, Lausanne, Switzerland.
Ann Transl Med. 2018 Aug;6(15):308. doi: 10.21037/atm.2018.07.02.
Multivariable regression models are widely used in medical literature for the purpose of diagnosis or prediction. Conventionally, the adequacy of these models is assessed using metrics of diagnostic performances such as sensitivity and specificity, which fail to account for clinical utility of a specific model. Decision curve analysis (DCA) is a widely used method to measure this utility. In this framework, a clinical judgment of the relative value of benefits (treating a true positive case) and harms (treating a false positive case) associated with prediction models is made. As such, the preferences of patients or policy-makers are accounted for by using a metric called threshold probability. A decision analytic measure called net benefit is then calculated for each possible threshold probability, which puts benefits and harms on the same scale. The article is a technical note on how to perform DCA in R environment. The decision curve is depicted with the system. Correction for overfitting is done via either bootstrap or cross-validation. Confidence interval and P values for the comparison of two models are calculated using bootstrap method. Furthermore, we describe a method for computing area under net benefit for the comparison of two models. The average deviation about the probability threshold (ADAPT), which is a more recently developed index to measure the utility of a prediction model, is also introduced in this article.
多变量回归模型在医学文献中被广泛用于诊断或预测目的。传统上,这些模型的充分性是使用诊断性能指标(如敏感性和特异性)来评估的,而这些指标未能考虑特定模型的临床效用。决策曲线分析(DCA)是一种广泛用于衡量这种效用的方法。在此框架中,对与预测模型相关的益处(治疗真正阳性病例)和危害(治疗假阳性病例)的相对价值进行临床判断。因此,通过使用一种称为阈值概率的指标来考虑患者或政策制定者的偏好。然后针对每个可能的阈值概率计算一种称为净效益的决策分析指标,该指标将益处和危害放在同一尺度上。本文是一篇关于如何在R环境中进行DCA的技术说明。决策曲线通过该系统进行描绘。通过自助法或交叉验证对过拟合进行校正。使用自助法计算两个模型比较的置信区间和P值。此外,我们描述了一种计算两个模型比较的净效益曲线下面积的方法。本文还介绍了关于概率阈值的平均偏差(ADAPT),这是一种最近开发的用于衡量预测模型效用的指标。