Feng Dai, Cortese Giuliana, Baumgartner Richard
1 Biometrics Research, Merck Research Lab, NJ, USA.
2 Department of Statistical Sciences, University of Padova, Italy.
Stat Methods Med Res. 2017 Dec;26(6):2603-2621. doi: 10.1177/0962280215602040. Epub 2015 Aug 30.
The receiver operating characteristic (ROC) curve is frequently used as a measure of accuracy of continuous markers in diagnostic tests. The area under the ROC curve (AUC) is arguably the most widely used summary index for the ROC curve. Although the small sample size scenario is common in medical tests, a comprehensive study of small sample size properties of various methods for the construction of the confidence/credible interval (CI) for the AUC has been by and large missing in the literature. In this paper, we describe and compare 29 non-parametric and parametric methods for the construction of the CI for the AUC when the number of available observations is small. The methods considered include not only those that have been widely adopted, but also those that have been less frequently mentioned or, to our knowledge, never applied to the AUC context. To compare different methods, we carried out a simulation study with data generated from binormal models with equal and unequal variances and from exponential models with various parameters and with equal and unequal small sample sizes. We found that the larger the true AUC value and the smaller the sample size, the larger the discrepancy among the results of different approaches. When the model is correctly specified, the parametric approaches tend to outperform the non-parametric ones. Moreover, in the non-parametric domain, we found that a method based on the Mann-Whitney statistic is in general superior to the others. We further elucidate potential issues and provide possible solutions to along with general guidance on the CI construction for the AUC when the sample size is small. Finally, we illustrate the utility of different methods through real life examples.
受试者工作特征（ROC）曲线常用于衡量诊断测试中连续指标的准确性。ROC曲线下面积（AUC）可以说是ROC曲线最广泛使用的汇总指标。尽管小样本量情况在医学测试中很常见，但文献中总体上缺少对各种构建AUC置信/可信区间（CI）方法的小样本量性质的全面研究。在本文中，我们描述并比较了29种在可用观测值数量较少时构建AUC置信区间的非参数和参数方法。所考虑的方法不仅包括那些已被广泛采用的，还包括那些较少被提及的，或者据我们所知从未应用于AUC情况的方法。为了比较不同方法，我们进行了一项模拟研究，数据来自具有相等和不相等方差的双正态模型以及具有各种参数和相等及不相等小样本量的指数模型。我们发现，真实AUC值越大且样本量越小，不同方法的结果差异就越大。当模型设定正确时，参数方法往往优于非参数方法。此外，在非参数领域，我们发现基于曼-惠特尼统计量的方法总体上优于其他方法。我们进一步阐明了潜在问题，并针对小样本量时AUC置信区间的构建提供了可能的解决方案以及一般指导。最后，我们通过实际例子说明了不同方法的实用性。
