Gur D, Rockette H E, Good W F, Slasky B S, Cooperstein L A, Straub W H, Obuchowski N A, Metz C E
Department of Biostatistics, University of Pittsburgh, Pennsylvania 15261.
Invest Radiol. 1990 Mar;25(3):230-4. doi: 10.1097/00004424-199003000-00004.
Receiver-operating characteristic (ROC) analysis has been used in many medical imaging applications during the past decade. In order to ensure that reader-confidence ratings are analyzable (well distributed to meet convergence requirements of curve-fitting algorithms) and meaningful (limit extrapolation of the data), many investigators train readers specifically for this purpose. No experimental data are available concerning the possible effects of such training on the results of ROC studies. We performed a multi-observer, multi-disease study in which 300 chest images were rated by four radiologists before and after they were trained to provide well-distributed confidence ratings. The results indicate that for our data set, reader and disease-specific accuracy was not significantly affected by the training process for interstitial disease and pneumothoraces. However, the accuracy of two readers was significantly affected for the detection of nodules (P less than 0.05), and the overall accuracy of one reader was significantly affected for the classification of normal versus abnormal images (P less than 0.01). Thus, in spite of the difficulties associated with the performance of ROC studies in a free-reading environment, one should carefully consider the possible effects of any intervention on the results prior to conducting ROC studies.
在过去十年中，接受者操作特征（ROC）分析已在许多医学成像应用中得到使用。为了确保读者置信度评分可分析（分布良好以满足曲线拟合算法的收敛要求）且有意义（限制数据的外推），许多研究者专门为此目的对读者进行培训。尚无关于此类培训对ROC研究结果可能产生的影响的实验数据。我们进行了一项多观察者、多疾病研究，其中300张胸部图像由四位放射科医生在接受提供分布良好的置信度评分的培训前后进行评分。结果表明，对于我们的数据集，间质性疾病和气胸的读者和疾病特异性准确性并未受到培训过程的显著影响。然而，在检测结节方面，两位读者的准确性受到显著影响（P小于0.05），在正常图像与异常图像分类方面，一位读者的总体准确性受到显著影响（P小于0.01）。因此，尽管在自由阅读环境中进行ROC研究存在困难，但在开展ROC研究之前，应仔细考虑任何干预措施对结果可能产生的影响。
