Wilding Gregory E, Consiglio Joseph D, Shan Guogen
Department of Biostatistics, The State University of New York at Buffalo, Buffalo, NY, USA.
Stat Med. 2014 Jul 30;33(17):2998-3012. doi: 10.1002/sim.6135. Epub 2014 Mar 17.
Testing involving the intra-class kappa coefficient is commonly performed in order to assess agreement involving categorical ratings. A number of procedures have been proposed, which make use of the limiting null distribution as the sample size goes to infinity in order to compute the observed significance. As with many tests based on asymptotic null distributions, these tests are associated with problematic type I error control for selected sample sizes and points in the parameter space. We propose and study a collection of exact testing approaches for both the one-sample and K-sample scenarios. For the one-sample case, p-values are obtained using the exact distribution of the test statistic conditional on a sufficient statistic. In addition, unconditional approaches are considered on the basis of maximization across the nuisance parameter space. Numerical evaluation reveals advantages with the exact unconditional procedures.
涉及组内kappa系数的测试通常用于评估分类评级的一致性。已经提出了许多程序，这些程序利用样本量趋于无穷大时的极限零分布来计算观察到的显著性。与许多基于渐近零分布的测试一样，这些测试在选定的样本量和参数空间中的点上与有问题的I型错误控制相关。我们针对单样本和K样本情况提出并研究了一系列精确测试方法。对于单样本情况，使用基于充分统计量的测试统计量的精确分布来获得p值。此外，基于在干扰参数空间上的最大化考虑了无条件方法。数值评估揭示了精确无条件程序的优势。
