Obuchowski N A, Lieber M L
Department of Biostatistics and Epidemiology, Cleveland Clinic Foundation, OH 44195-5196, USA.
Acad Radiol. 1998 Aug;5(8):561-71. doi: 10.1016/s1076-6332(98)80208-0.
The authors performed this study to address two practical questions. First, how large does the sample size need to be for confidence intervals (CIs) based on the usual asymptotic methods to be appropriate? Second, when the sample size is smaller than this threshold, what alternative method of CI construction should be used?
The authors performed a Monte Carlo simulation study where 95% CIs were constructed for the receiver operating characteristic (ROC) area and for the difference between two ROC areas for rating and continuous test results--for ROC areas of moderate and high accuracy--by using both parametric and nonparametric estimation methods. Alternative methods evaluated included several bootstrap CIs and CIs with the Student t distribution.
For the difference between two ROC areas, CIs based on the asymptotic theory provided adequate coverage even when the sample size was very small (20 patients). In contrast, for a single ROC area, the asymptotic methods do not provide adequate CI coverage for small samples; for ROC areas of high accuracy, the sample size must be large (more than 200 patients) for the asymptotic methods to be applicable. The recommended alternative (bootstrap percentile, bootstrap t, or bootstrap bias-corrected accelerated method) depends on the estimation approach, format of the test results, and ROC area.
Currently, there is not a single best alternative for constructing CIs for a single ROC area for small samples.
作者开展本研究以解决两个实际问题。其一，基于常用渐近方法的置信区间（CI）要适用的话，样本量需要多大？其二，当样本量小于该阈值时，应使用何种替代的CI构建方法？
作者进行了一项蒙特卡洛模拟研究，通过参数估计和非参数估计方法，针对中等和高精度的受试者工作特征（ROC）曲线下面积以及评分和连续检验结果的两个ROC曲线下面积之差构建95%置信区间。评估的替代方法包括几种自助法置信区间和基于学生t分布的置信区间。
对于两个ROC曲线下面积之差，即使样本量非常小（20例患者），基于渐近理论的置信区间也能提供足够的覆盖范围。相比之下，对于单个ROC曲线下面积，渐近方法对于小样本不能提供足够的置信区间覆盖范围；对于高精度的ROC曲线下面积，渐近方法要适用的话样本量必须很大（超过200例患者）。推荐的替代方法（自助百分位数法、自助t法或自助偏差校正加速法）取决于估计方法、检验结果的形式以及ROC曲线下面积。
目前，对于小样本单个ROC曲线下面积构建置信区间，不存在单一的最佳替代方法。
