Comparison of receiver operating characteristic curves on the basis of optimal operating points.

RATIONALE AND OBJECTIVES

We developed a method of comparing receiver operating characteristic (ROC) curves on the basis of the utilities associated with their optimal operating points (OOPs).

METHODS

OOPs were computed for paired ROC curves on the basis of isocost lines in ROC space with slopes ranging from 0.1 to 3.0. For each pair of OOPs corresponding to a single isocost slope, the difference in costs and the variance of this difference was computed. A sensitivity analysis was thus obtained for the difference between the two curves over a range of isocost slopes. Three published data sets were evaluated using this technique, as well as by comparisons of areas under the curves and of true-positive fractions at fixed false-positive fractions.

RESULTS

The OOPs of paired ROC curves often occur at different false-positive fractions. Comparisons of ROC curves on the basis of OOPs may provide results that differ from comparisons of curves at a fixed false-positive fraction.

CONCLUSION

ROC curves may be compared on the basis of utilities associated with their OOPs. This comparison of the optimal performance of two diagnostic tests may differ from conventional statistical comparisons.