Fuhrman Carl R, Britton Cynthia A, Bender Thomas, Sumkin Jules H, Brown Manuel L, Holbert J Michael, Chang Thomas S, Rockette Howard E, Gur David
Department of Radiology, University of Pittsburgh, 200 Lothrop St., Pittsburgh, PA 15213-2582, USA.
AJR Am J Roentgenol. 2002 Dec;179(6):1551-3. doi: 10.2214/ajr.179.6.1791551.
We used receiver operating characteristic (ROC) analysis to compare two methods of evaluating observer performance in detecting an abnormality on chest radiographs. In the first method, the abnormality in question, rib fracture, was one of five investigated, and it was the only one of interest in the second.
Eight experienced observers viewed 117 posteroanterior chest radiographs in two interpretation modes. Fifty-four of these images depicted rib fractures that had been rated as subtle for detection. The likelihood of the presence of a rib fracture was rated as one of five abnormalities in question in one mode and the sole abnormality of interest in the other mode.
Six of the observers performed better during the single-abnormality mode, one performed equally well in both modes, and one performed better during the multiple-abnormality mode. The average area under the ROC curves (A(z)) was 0.73 +/- 0.07 for the multiple-abnormality mode and 0.80 +/- 0.04 for the single-abnormality mode. The results were significantly different (p < 0.05).
Study methodology can significantly affect the results in ROC studies, particularly for abnormalities that may not be perceived as primary or important. The order in which abnormalities appear on a checklist report form may be important.
我们采用受试者工作特征（ROC）分析来比较两种评估观察者在胸部X光片上检测异常表现的方法。在第一种方法中，所讨论的异常情况——肋骨骨折，是五项被研究的异常之一，而在第二种方法中，它是唯一感兴趣的异常。
八位经验丰富的观察者以两种解读模式查看了117张后前位胸部X光片。其中54张图像显示有肋骨骨折，这些骨折被评定为难以检测。在一种模式下，肋骨骨折存在的可能性被评定为所讨论的五种异常之一，而在另一种模式下，它是唯一感兴趣的异常。
六位观察者在单一异常模式下表现更好，一位在两种模式下表现相同，一位在多种异常模式下表现更好。多种异常模式下ROC曲线下的平均面积（A(z)）为0.73±0.07，单一异常模式下为0.80±0.04。结果有显著差异（p<0.05）。
研究方法可显著影响ROC研究的结果，特别是对于那些可能不被视为主要或重要的异常情况。异常情况在清单报告表上出现的顺序可能很重要。
