Faculty of Health Medicine and Life Sciences, Department of Methodology and Statistics, Care and Public Health Research Institute (CAPHRI), Maastricht University, Limburg, The Netherlands.
Department of Statistics, Computer Science, Applications "Giuseppe Parenti", The University of Florence, Italy.
Stat Methods Med Res. 2024 Mar;33(3):532-553. doi: 10.1177/09622802231224657. Epub 2024 Feb 6.
Reliability of measurement instruments providing quantitative outcomes is usually assessed by an intraclass correlation coefficient. When participants are repeatedly measured by a single rater or device, or, are each rated by a different group of raters, the intraclass correlation coefficient is based on a one-way analysis of variance model. When planning a reliability study, it is essential to determine the number of participants and measurements per participant (i.e. number of raters or number of repeated measurements). Three different sample size determination approaches under the one-way analysis of variance model were identified in the literature, all based on a confidence interval for the intraclass correlation coefficient. Although eight different confidence interval methods can be identified, Wald confidence interval with Fisher's large sample variance approximation remains most commonly used despite its well-known poor statistical properties. Therefore, a first objective of this work is comparing the statistical properties of all identified confidence interval methods-including those overlooked in previous studies. A second objective is developing a general procedure to determine the sample size using all approaches since a closed-form formula is not always available. This procedure is implemented in an R Shiny app. Finally, we provide advice for choosing an appropriate sample size determination method when planning a reliability study.
测量仪器提供定量结果的可靠性通常通过组内相关系数进行评估。当参与者被单个评分者或设备重复测量，或者由不同的评分者小组进行评分时，组内相关系数基于方差分析模型。在计划可靠性研究时，确定参与者数量和每个参与者的测量次数（即评分者数量或重复测量次数）至关重要。文献中确定了方差分析模型下的三种不同的样本量确定方法，均基于组内相关系数的置信区间。尽管可以确定八种不同的置信区间方法，但 Wald 置信区间与 Fisher 大样本方差逼近仍然是最常用的方法，尽管它具有众所周知的较差的统计特性。因此，这项工作的第一个目标是比较所有确定的置信区间方法的统计特性，包括之前研究中忽略的方法。第二个目标是开发一种使用所有方法确定样本量的通用程序，因为并非总是提供封闭形式的公式。该程序在 R Shiny 应用程序中实现。最后，我们在计划可靠性研究时提供了选择适当的样本量确定方法的建议。
