Hwang Yi-Ting, Hsu Ya-Ru, Su Nan-Cheng
Department of Statistics, National Taipei University, Sancia, New Taipei City, Taiwan.
J Appl Stat. 2024 Jul 14;52(3):555-577. doi: 10.1080/02664763.2024.2374931. eCollection 2025.
One of the main objectives of disease prevention is to lower the healthcare costs and improve the quality of life. To achieve this, reliable diagnostic tools are needed. The diagnostic performance of a tool can be measured by the ROC curve and the AUC. However, some diagnostic tools such as MRI images are not objective, but depend on the interpretation of experts. Therefore, the accuracy of these tools may vary depending on who is interpreting them. To account for possible correlations when multiple readers collect data, Dorfman, Berbaum and Metz (1992) proposed using AUC pseudovalues from the jackknife sampling method and applying them to the mixed model to analyze the diagnostic reagent's accuracy. However, pseudovalues may go beyond the AUC range. Also, the random effect estimate may be negative due to a small number of readers. This paper develops tests based on AUC estimates and gives their asymptotic distribution. Moreover, a two-stage test is suggested to correct for negative random effect estimates. Four tests are created in total and their performance is evaluated by Monte Carlo simulations. The distributional assumption's robustness of these tests is checked, and their applicability is demonstrated by two real data sets.
疾病预防的主要目标之一是降低医疗成本并提高生活质量。为实现这一目标,需要可靠的诊断工具。工具的诊断性能可以通过ROC曲线和AUC来衡量。然而,一些诊断工具,如MRI图像,并不客观,而是依赖于专家的解读。因此,这些工具的准确性可能因解读人员的不同而有所差异。为了在多个读者收集数据时考虑可能的相关性,多尔夫曼、伯鲍姆和梅茨(1992年)提出使用刀切抽样法的AUC伪值,并将其应用于混合模型以分析诊断试剂的准确性。然而,伪值可能超出AUC范围。此外,由于读者数量较少,随机效应估计可能为负。本文基于AUC估计值开发了检验方法,并给出了它们的渐近分布。此外,还建议采用两阶段检验来校正负随机效应估计值。总共创建了四个检验,并通过蒙特卡罗模拟评估了它们的性能。检查了这些检验的分布假设的稳健性,并通过两个真实数据集证明了它们的适用性。