Statistics Department, 8367The George Washington University, Washington, USA.
4137U.S. Food and Drug Administration, CDRH, OSEL, DIDSR, Silver Spring, USA.
Stat Methods Med Res. 2022 Nov;31(11):2069-2086. doi: 10.1177/09622802221111539. Epub 2022 Jul 5.
The area under the receiver operating characteristic curve (AUC) is widely used in evaluating diagnostic performance for many clinical tasks. It is still challenging to evaluate the reading performance of distinguishing between positive and negative regions of interest (ROIs) in the nested-data problem, where multiple ROIs are nested within the cases. To address this issue, we identify two kinds of AUC estimators, within-cases AUC and between-cases AUC. We focus on the between-cases AUC estimator, since our main research interest is in patient-level diagnostic performance rather than location-level performance (the ability to separate ROIs with and without disease within each patient). Another reason is that as the case number increases, the number of between-cases paired ROIs is much larger than the number of within-cases ROIs. We provide estimators for the variance of the between-cases AUC and for the covariance when there are two readers. We derive and prove the above estimators' theoretical values based on a simulation model and characterize their behavior using Monte Carlo simulation results. We also provide a real-data example. Moreover, we connect the distribution-based simulation model with the simulation model based on the linear mixed-effect model, which helps better understand the sources of variation in the simulated dataset.
受试者工作特征曲线下面积（AUC）广泛用于评估许多临床任务的诊断性能。在嵌套数据问题中，评估区分感兴趣区域（ROI）阳性和阴性的阅读性能仍然具有挑战性，其中多个 ROI 嵌套在病例中。为了解决这个问题，我们确定了两种 AUC 估计量，即病例内 AUC 和病例间 AUC。我们专注于病例间 AUC 估计量，因为我们的主要研究兴趣是患者水平的诊断性能，而不是位置水平的性能（在每个患者中分离有和无疾病的 ROI 的能力）。另一个原因是，随着病例数量的增加，病例间配对 ROI 的数量远远大于病例内 ROI 的数量。当有两个读者时，我们为病例间 AUC 的方差和协方差提供了估计量。我们基于模拟模型推导出并证明了上述估计量的理论值，并使用蒙特卡罗模拟结果描述了它们的行为。我们还提供了一个真实数据示例。此外，我们将基于分布的模拟模型与基于线性混合效应模型的模拟模型联系起来，这有助于更好地理解模拟数据集的变异来源。
