Gwet Kilem L
Advanced Analytics, LLC, Gaithersburg, MD, USA.
Educ Psychol Meas. 2016 Aug;76(4):609-637. doi: 10.1177/0013164415596420. Epub 2015 Jul 28.
This article addresses the problem of testing the difference between two correlated agreement coefficients for statistical significance. A number of authors have proposed methods for testing the difference between two correlated kappa coefficients, which require either the use of resampling methods or the use of advanced statistical modeling techniques. In this article, we propose a technique similar to the classical pairwise test for means, which is based on a large-sample linear approximation of the agreement coefficient. We illustrate the use of this technique with several known agreement coefficients including Cohen's kappa, Gwet's AC, Fleiss's generalized kappa, Conger's generalized kappa, Krippendorff's alpha, and the Brenann-Prediger coefficient. The proposed method is very flexible, can accommodate several types of correlation structures between coefficients, and requires neither advanced statistical modeling skills nor considerable computer programming experience. The validity of this method is tested with a Monte Carlo simulation.
本文探讨了检验两个相关一致性系数之间差异的统计学显著性问题。许多作者提出了检验两个相关kappa系数之间差异的方法，这些方法要么需要使用重采样方法，要么需要使用先进的统计建模技术。在本文中，我们提出了一种类似于经典均值成对检验的技术，该技术基于一致性系数的大样本线性近似。我们用几个已知的一致性系数说明了该技术的应用，包括科恩kappa系数、格韦特AC系数、弗莱斯广义kappa系数、康格广义kappa系数、克里彭多夫alpha系数和布伦南 - 普雷迪格系数。所提出的方法非常灵活，可以适应系数之间的几种相关结构，既不需要先进的统计建模技能，也不需要大量的计算机编程经验。通过蒙特卡罗模拟检验了该方法的有效性。
