Statistical comparison of ROC curves from multiple readers.

Receiver operating characteristic (ROC) analysis is the commonly accepted method for comparing diagnostic imaging systems. In general, ROC studies are designed in such a way that multiple readers read the same images and each image is presented by means of two different imaging systems. Statistical methods for the comparison of the ROC curves from one reader have been developed, but extension of these methods to multiple readers is not straightforward. A new method of analysis is presented for the comparison of ROC curves from multiple readers. This method includes a non-parametric estimation of the variances and covariances between the various areas under the curves. The method described is more appropriate than the paired t test, because it also takes the case-sample variation into account.