Brilliant L B, Lepkowski J M, Musch D C
Am J Epidemiol. 1983 Aug;118(2):265-79. doi: 10.1093/oxfordjournals.aje.a113633.
In the Nepal Blindness Survey, 39,887 people in 105 sites were examined by 10 ophthalmologists from Nepal and four other countries during 1981. Ophthalmic protocols were pretested on approximately 3000 subjects; however, interobserver variability was inevitable. To quantify the amount of variability and assess the reliability of important ophthalmic measures, a study of interobserver agreement was conducted. Five ophthalmologists, randomly assigned to one of two examining stations in a single survey site, carried out 529 pairs of examinations. Eighty demographic and ophthalmic variables were assessed at each station. In 62 of 80 (77.5%) measures, observer agreement exceeded 90%. Since pathologic findings were rare, however, chance agreement alone could yield misleadingly high per cent agreement; therefore, the kappa statistic was used for assessing comparative reliability of ophthalmic measures. There were 74 measures for which kappa could be computed and ranked by strength of agreement: 20 (27%) showed excellent agreement (kappa = 0.75-1.00), 39 (53%) showed fair to good agreement (kappa = 0.40-0.74), and 15 (20%) showed poor agreement (kappa less than 0.40). In general, measures dealing with blindness prevalence or causes of blindness showed substantial or almost perfect agreement, while polychotomous descriptions of rare clinical signs demonstrated less agreement.
在尼泊尔失明调查中，1981年，来自尼泊尔及其他四个国家的10位眼科医生在105个地点对39,887人进行了检查。眼科检查方案在大约3000名受试者身上进行了预测试；然而，观察者间的变异性是不可避免的。为了量化变异性的程度并评估重要眼科测量的可靠性，开展了一项观察者间一致性研究。在一个调查地点，随机分配到两个检查站点之一的5位眼科医生进行了529对检查。每个站点评估了80个人口统计学和眼科变量。在80项测量中的62项（77.5%）中，观察者间的一致性超过了90%。然而，由于病理结果很少见，仅靠偶然一致性可能会产生误导性的高百分比一致性；因此，kappa统计量用于评估眼科测量的相对可靠性。有74项测量可以计算kappa值并按一致性强度进行排序：20项（27%）显示出极好的一致性（kappa = 0.75 - 1.00），39项（53%）显示出中等至良好的一致性（kappa = 0.40 - 0.74），15项（20%）显示出较差的一致性（kappa小于0.40）。一般来说，涉及失明患病率或失明原因的测量显示出高度或几乎完美的一致性，而对罕见临床体征的多分类描述显示出的一致性较低。
