Welch L S, Hunting K L, Balmes J, Bresnitz E A, Guidotti T L, Lockey J E, Myo-Lwin T
Department of Occupational/Environmental Medicine, Washington Hospital Center, Washington, DC 20010-2975, USA.
Chest. 1998 Dec;114(6):1740-8. doi: 10.1378/chest.114.6.1740.
This study describes the extent of agreement in classification of chest radiographs using the International Labor Organization (ILO) classification among six readers from the United States and Canada. A set of 119 radiographs was created and read by three Canadian and three US readers. The two ratings of interest were profusion (scored from 0/- to 3/+) and pleural abnormalities consistent with pneumoconiosis (scored with the ILO system, then collapsed into a yes/no). We used a number of approaches to evaluate interreader agreement on profusion and pleural changes, determining concordance, observed agreement, kappa statistic, and a new measure to approximate sensitivity and specificity. This study found that five of six readers had good fair to good agreement for pleural findings and for profusion as a dichotomous variable (> or = 1/0 vs < or = 0/1) using the kappa statistic, while a sixth reader had poor agreement. We found that concordance, expressed as percent agreement, was higher for normal radiographs than for ones that showed disease, and describe the use of the kappa statistic to control for this finding. This analysis adds to the existing literature with the use of the kappa statistic, and by presenting a new measure for "underreading" and "overreading" tendencies.
本研究描述了来自美国和加拿大的六位读者使用国际劳工组织（ILO）分类法对胸部X光片进行分类时的一致程度。创建了一组119张X光片，由三位加拿大读者和三位美国读者阅读。两个感兴趣的评级分别是肺野病变程度（从0/-到3/+评分）和与尘肺病一致的胸膜异常（用ILO系统评分，然后合并为是/否）。我们使用了多种方法来评估读者间在肺野病变程度和胸膜变化方面的一致性，确定一致性、观察到的一致性、kappa统计量，以及一种近似灵敏度和特异度的新方法。本研究发现，使用kappa统计量时，六位读者中有五位在胸膜表现和作为二分变量的肺野病变程度（>或=1/0对<或=0/1）方面有较好至良好的一致性，而第六位读者的一致性较差。我们发现，以一致百分比表示的一致性，正常X光片高于显示疾病的X光片，并描述了使用kappa统计量来控制这一发现。本分析通过使用kappa统计量以及提出一种针对“漏读”和“过度阅读”倾向的新方法，为现有文献增添了内容。
