科恩氏 κ系数与格瓦特氏 AC1 系数在计算评定者间信度系数时的比较：一项对人格障碍样本进行的研究。
A comparison of Cohen's Kappa and Gwet's AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples.
机构信息
Department of Psychiatry, Faculty of Medicine, Chiang Mai University, Chiang Mai 50200, Thailand.
出版信息
BMC Med Res Methodol. 2013 Apr 29;13:61. doi: 10.1186/1471-2288-13-61.
BACKGROUND
Rater agreement is important in clinical research, and Cohen's Kappa is a widely used method for assessing inter-rater reliability; however, there are well documented statistical problems associated with the measure. In order to assess its utility, we evaluated it against Gwet's AC1 and compared the results.
METHODS
This study was carried out across 67 patients (56% males) aged 18 to 67, with a mean SD of 44.13 ± 12.68 years. Nine raters (7 psychiatrists, a psychiatry resident and a social worker) participated as interviewers, either for the first or the second interviews, which were held 4 to 6 weeks apart. The interviews were held in order to establish a personality disorder (PD) diagnosis using DSM-IV criteria. Cohen's Kappa and Gwet's AC1 were used and the level of agreement between raters was assessed in terms of a simple categorical diagnosis (i.e., the presence or absence of a disorder). Data were also compared with a previous analysis in order to evaluate the effects of trait prevalence.
RESULTS
Gwet's AC1 was shown to have higher inter-rater reliability coefficients for all the PD criteria, ranging from .752 to 1.000, whereas Cohen's Kappa ranged from 0 to 1.00. Cohen's Kappa values were high and close to the percentage of agreement when the prevalence was high, whereas Gwet's AC1 values appeared not to change much with a change in prevalence, but remained close to the percentage of agreement. For example a Schizoid sample revealed a mean Cohen's Kappa of .726 and a Gwet's AC1of .853 , which fell within the different level of agreement according to criteria developed by Landis and Koch, and Altman and Fleiss.
CONCLUSIONS
Based on the different formulae used to calculate the level of chance-corrected agreement, Gwet's AC1 was shown to provide a more stable inter-rater reliability coefficient than Cohen's Kappa. It was also found to be less affected by prevalence and marginal probability than that of Cohen's Kappa, and therefore should be considered for use with inter-rater reliability analysis.
背景
在临床研究中，评分者间一致性很重要，Cohen's Kappa 是一种广泛用于评估评分者间可靠性的方法；然而，该方法存在许多有文献记录的统计学问题。为了评估其效用，我们将其与 Gwet 的 AC1 进行了对比。
方法
本研究共纳入 67 例患者（男性占 56%），年龄 18 至 67 岁，平均年龄（标准差）为 44.13 ± 12.68 岁。9 名评分者（7 名精神科医生、1 名精神科住院医师和 1 名社会工作者）作为访谈者参与研究，他们分别进行了首次或第二次访谈，两次访谈间隔 4 至 6 周。访谈是根据 DSM-IV 标准进行人格障碍（PD）诊断。使用 Cohen's Kappa 和 Gwet 的 AC1 评估评分者间的一致性，并根据简单的分类诊断（即存在或不存在障碍）评估评分者间的一致性水平。还将数据与之前的分析进行了比较，以评估特征患病率的影响。
结果
对于所有 PD 标准，Gwet 的 AC1 显示出更高的评分者间可靠性系数，范围从 0.752 到 1.000，而 Cohen's Kappa 范围从 0 到 1.00。当患病率较高时，Cohen's Kappa 值较高且接近一致性百分比，而 Gwet 的 AC1 值似乎不会随患病率的变化而变化，但仍接近一致性百分比。例如，一个分裂样样本显示 Cohen's Kappa 的平均值为 0.726，Gwet 的 AC1 为 0.853，根据 Landis 和 Koch 以及 Altman 和 Fleiss 制定的标准，这两个值都处于不同的一致性水平。
结论
基于计算机会校正一致性水平的不同公式，Gwet 的 AC1 显示出比 Cohen's Kappa 更稳定的评分者间可靠性系数。与 Cohen's Kappa 相比，它受患病率和边缘概率的影响更小，因此应考虑用于评分者间可靠性分析。
Zotero 插件