Jin Mei, Liu Aiyi, Chen Zhen, Li Zhaohai
George Washington University ; Capital One.
National Institute of Child Health and Human Development.
Stat Sin. 2013 Jul;23(4):1743-1759. doi: 10.5705/ss.2012.036s.
Inter-rater reliability is usually assessed by means of the intraclass correlation coefficient. Using two-way analysis of variance to model raters and subjects as random effects, we derive group sequential testing procedures for the design and analysis of reliability studies in which multiple raters evaluate multiple subjects. Compared with the conventional fixed sample procedures, the group sequential test has smaller average sample number. The performance of the proposed technique is examined using simulation studies and critical values are tabulated for a range of two-stage design parameters. The methods are exemplified using data from the Physician Reliability Study for diagnosis of endometriosis.
评分者间信度通常通过组内相关系数来评估。使用双向方差分析将评分者和受试者建模为随机效应，我们推导了用于多评分者评估多受试者的信度研究设计和分析的序贯检验程序。与传统的固定样本程序相比，序贯检验的平均样本量更小。通过模拟研究检验了所提出技术的性能，并列出了一系列两阶段设计参数的临界值。使用来自子宫内膜异位症诊断的医生信度研究的数据对这些方法进行了举例说明。
