Department of Biomedical Engineering, Hospital of Halland, Varberg SE-432 81, Sweden.
Department of Radiation Physics, Institute of Clinical Sciences, The Sahlgrenska Academy at University of Gothenburg, Gothenburg SE-413 45, Sweden.
Radiat Prot Dosimetry. 2021 Oct 12;195(3-4):378-390. doi: 10.1093/rpd/ncab066.
The purpose of the present work was to evaluate the use of resampling statistical methods for analysis of visual grading data-implemented in the software VGC Analyzer-by comparing the reanalyzed results from previously performed visual grading studies with the results calculated by gold standard receiver operating characteristic (ROC) methodology, Obuchowski-Rockette (OR)-Dorfman-Berbaum-Metz (DBM) multiple-readers and multiple-case (MRMC) and by analysis of simulated visual grading data where the true distribution was presumed to be known. The reanalysis was performed on two multiple-reader studies with non-paired data and paired data, respectively. The simulation study was performed by simulating a large number of visual grading characteristics (VGC) studies and by analyzing the statistical distribution of null hypothesis (H0) rejection rate. The comparison with OR-DBM MRMC showed good agreement when analyzing non-paired data for both fixed-reader and random-reader settings for the calculated area under the curve values and the confidence intervals (CIs). For paired data analysis, VGC Analyzer showed significantly lower CIs compared with the ROC software. This effect was also illustrated by the simulation study, where the VGC Analyzer, in general, showed good accuracy for simulated studies with stable statistical basis. For simulated studies with unstable statistics, the accuracy in the H0 rejection rate decreased. The present study has shown that resampling methodology can be used to accurately perform the statistical analysis of a VGC study, although the resampling technique used makes the method sensitive to small data sets.
本研究旨在评估重采样统计方法在分析视觉分级数据(VGC)中的应用，该方法由 VGC Analyzer 软件实现，我们通过比较之前执行的视觉分级研究的重新分析结果与金标准接收器工作特征(ROC)方法、Obuchowski-Rockette (OR)-Dorfman-Berbaum-Metz (DBM)多读者和多病例(MRMC)以及模拟视觉分级数据的分析结果，来实现对该方法的评估，这些模拟数据的真实分布被假定为已知。重新分析分别针对具有非配对数据和配对数据的两个多读者研究进行。通过模拟大量视觉分级特征(VGC)研究并分析零假设(H0)拒绝率的统计分布来进行模拟研究。当分析非配对数据时，对于固定读者和随机读者设置，VGC Analyzer 与 OR-DBM MRMC 分析计算得到的曲线下面积值和置信区间(CIs)时，具有良好的一致性。对于配对数据分析，VGC Analyzer 与 ROC 软件相比，显示出显著更低的置信区间。该效果也通过模拟研究得到了说明，VGC Analyzer 通常在具有稳定统计基础的模拟研究中具有良好的准确性。对于具有不稳定统计数据的模拟研究，H0 拒绝率的准确性会降低。本研究表明，重采样方法可用于准确执行 VGC 研究的统计分析，尽管所使用的重采样技术使该方法对小数据集较为敏感。
