Department of Future Technologies, University of Turku, Turku, Finland.
Department of Urology, Turku University Hospital, Turku, Finland.
Stat Methods Med Res. 2019 Oct-Nov;28(10-11):2975-2991. doi: 10.1177/0962280218795190. Epub 2018 Aug 20.
Receiver operating characteristic analysis is widely used for evaluating diagnostic systems. Recent studies have shown that estimating an area under receiver operating characteristic curve with standard cross-validation methods suffers from a large bias. The leave-pair-out cross-validation has been shown to correct this bias. However, while leave-pair-out produces an almost unbiased estimate of area under receiver operating characteristic curve, it does not provide a ranking of the data needed for plotting and analyzing the receiver operating characteristic curve. In this study, we propose a new method called tournament leave-pair-out cross-validation. This method extends leave-pair-out by creating a tournament from pair comparisons to produce a ranking for the data. Tournament leave-pair-out preserves the advantage of leave-pair-out for estimating area under receiver operating characteristic curve, while it also allows performing receiver operating characteristic analyses. We have shown using both synthetic and real-world data that tournament leave-pair-out is as reliable as leave-pair-out for area under receiver operating characteristic curve estimation and confirmed the bias in leave-one-out cross-validation on low-dimensional data. As a case study on receiver operating characteristic analysis, we also evaluate how reliably sensitivity and specificity can be estimated from tournament leave-pair-out receiver operating characteristic curves.
受试者工作特征分析被广泛用于评估诊断系统。最近的研究表明，使用标准的交叉验证方法估计受试者工作特征曲线下面积会存在较大的偏差。留对交叉验证已被证明可以纠正这种偏差。然而，虽然留对交叉验证可以对受试者工作特征曲线下面积进行几乎无偏的估计，但它不能提供绘制和分析受试者工作特征曲线所需的数据排名。在本研究中，我们提出了一种称为锦标赛留对交叉验证的新方法。该方法通过从配对比较中创建锦标赛来扩展留对交叉验证，从而为数据提供排名。锦标赛留对交叉验证保留了留对交叉验证在估计受试者工作特征曲线下面积方面的优势，同时也允许进行受试者工作特征分析。我们已经使用合成数据和真实世界数据表明，锦标赛留对交叉验证在估计受试者工作特征曲线下面积方面与留对交叉验证一样可靠，并证实了在低维数据上的留一交叉验证的偏差。作为受试者工作特征分析的案例研究，我们还评估了如何从锦标赛留对交叉验证的受试者工作特征曲线中可靠地估计灵敏度和特异性。
