Chen Weijie, Gong Qi, Gallas Brandon D
Food and Drug Administration, Center for Devices and Radiological Health, Office of Science and Engineering Laboratories, Division of Imaging, Diagnostics, and Software Reliability, Silver Spring, Maryland, United States.
J Med Imaging (Bellingham). 2018 Jul;5(3):031410. doi: 10.1117/1.JMI.5.3.031410. Epub 2018 May 17.
The widely used multireader multicase ROC study design for comparing imaging modalities is the fully crossed (FC) design: every reader reads every case of both modalities. We investigate paired split-plot (PSP) designs that may allow for reduced cost and increased flexibility compared with the FC design. In the PSP design, case images from two modalities are read by the same readers, thereby the readings are paired across modalities. However, within each modality, not every reader reads every case. Instead, both the readers and the cases are partitioned into a fixed number of groups and each group of readers reads its own group of cases-a split-plot design. Using a [Formula: see text]-statistic based variance analysis for AUC (i.e., area under the ROC curve), we show analytically that precision can be gained by the PSP design as compared with the FC design with the same number of readers and readings. Equivalently, we show that the PSP design can achieve the same statistical power as the FC design with a reduced number of readings. The trade-off for the increased precision in the PSP design is the cost of collecting a larger number of truth-verified patient cases than the FC design. This means that one can trade-off between different sources of cost and choose a least burdensome design. We provide a validation study to show the iMRMC software can be reliably used for analyzing data from both FC and PSP designs. Finally, we demonstrate the advantages of the PSP design with a reader study comparing full-field digital mammography with screen-film mammography.
用于比较成像模式的广泛使用的多读者多病例ROC研究设计是完全交叉（FC）设计：每个读者阅读两种模式的每个病例。我们研究了配对裂区（PSP）设计，与FC设计相比，它可能具有成本降低和灵活性提高的特点。在PSP设计中，来自两种模式的病例图像由相同的读者阅读，因此读数在不同模式之间是配对的。然而，在每种模式内，并非每个读者阅读每个病例。相反，读者和病例都被分成固定数量的组，每组读者阅读其自己组的病例——一种裂区设计。使用基于AUC（即ROC曲线下面积）的F统计量方差分析，我们通过分析表明，与具有相同读者数量和读数的FC设计相比，PSP设计可以提高精度。等效地，我们表明PSP设计可以在减少读数数量的情况下实现与FC设计相同的统计功效。PSP设计中精度提高的权衡是比FC设计收集更多经真值验证的患者病例的成本。这意味着可以在不同的成本来源之间进行权衡，并选择负担最小的设计。我们提供了一项验证研究，以表明iMRMC软件可以可靠地用于分析来自FC和PSP设计的数据。最后，我们通过一项读者研究展示了PSP设计的优势，该研究比较了全视野数字乳腺摄影与屏-片乳腺摄影。
