Resampling techniques in the analysis of non-binormal ROC data.

The methods most commonly used for analyzing receiver operating characteristic (ROC) data incorporate "binormal" assumptions about the latent frequency distributions of test results. Although these assumptions have proved robust to a wide variety of actual frequency distributions, some data sets do not "fit" the binormal model. In such cases, resampling techniques such as the jackknife and the bootstrap provide versatile, distribution-independent, and more appropriate methods for hypothesis testing. This article describes the application of resampling techniques to ROC data for which the binormal assumptions are not appropriate, and suggests that the bootstrap may be especially helpful in determining confidence intervals from small data samples. The widespread availability of ever-faster computers has made resampling methods increasingly accessible and convenient tools for data analysis.