A generalization of the one-sided two-sample Kolmogorov-Smirnov statistic for evaluating diagnostic tests.

Suppose a continuous diagnostic measurement is used to classify patients, and suppose E1 false negative errors and E2 false positive errors result. The quantities E1 and E2, and the total number of misclassifications, L = E1 + E2, depend on the choice of cut-off value. We have determined the null distribution of min L, where minimization is over all possible cut-off values. The statistic, min L, can be used as a quick one-sided two-sample test, and min L is also useful for evaluating publications which present only a 2 X 2 table of false positives, false negatives, true positives and true negatives. In such cases, one can use min L to assess the usefulness of the diagnostic measurement, even if one suspects that the authors chose that particular cut-off value which minimized L after looking at the data. We extend these results to a more general weighted loss L = vE1 + MUE2 where v and mu are positive integers, and we show that min L is a generalization of the one-sided two-sample Kolmogorov-Smirnov statistic, and, indeed, exactly equivalent to that statistic for appropriate choices of v and mu.