Centre for Educational Measurement at the University of Oslo (CEMO), Faculty of Educational Sciences, University of Oslo.
School of Education, Murdoch University.
Psychol Methods. 2020 Dec;25(6):747-775. doi: 10.1037/met0000261. Epub 2020 Mar 5.
Reporting the reliability of the scores obtained from a scale or test is part of the standard repertoire of empirical studies in psychology. With reliability being a key concept in psychometrics, researchers have become more and more interested in evaluating reliability coefficients across studies and, ultimately, quantify and explain possible between-study variation. This approach-commonly known as "reliability generalization"-can be specified within the framework of meta-analysis. The existing procedures of reliability generalization, however, have several methodological issues: (a) unrealistic and often untested assumptions on the measurement model underlying the reliability coefficients (e.g., essential τ-equivalence for Cronbach's α); (b) the use of univariate approaches to synthesizing reliabilities of total and subscale scores; (c) the lack of comparability across different types of reliability coefficients. However, these issues can be addressed directly through meta-analytic structural equation modeling (MASEM)-a method that combines meta-analysis with structural equation modeling through synthesizing either correlation matrices or model parameters across studies. The primary objective of this article is to present the potential MASEM has for the meta-analysis of reliability coefficients. We review the extant body of literature on the use of reliability generalization, discuss and illustrate two MASEM approaches (i.e., correlation-based and parameter-based MASEM), and propose some practical guidelines. Future directions for utilizing MASEM for reliability generalization are discussed. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
报告从量表或测试中获得的分数的可靠性是心理学实证研究标准内容的一部分。由于可靠性是心理测量学中的一个关键概念,研究人员越来越有兴趣跨研究评估可靠性系数,并最终量化和解释可能存在的研究间差异。这种方法——通常称为“可靠性概括”——可以在元分析框架内进行指定。然而,现有的可靠性概括程序存在几个方法学问题:(a) 对可靠性系数背后的测量模型的不切实际且经常未经检验的假设(例如,Cronbach's α 的基本 τ 等价性);(b) 对综合总分数和子分数可靠性的单变量方法的使用;(c) 不同类型的可靠性系数之间缺乏可比性。然而,这些问题可以通过元分析结构方程建模(MASEM)直接解决——一种通过在研究之间综合相关矩阵或模型参数将元分析与结构方程建模结合起来的方法。本文的主要目的是展示 MASEM 在可靠性系数元分析中的潜力。我们回顾了关于使用可靠性概括的现有文献,讨论并说明了两种 MASEM 方法(即基于相关的和基于参数的 MASEM),并提出了一些实用指南。讨论了利用 MASEM 进行可靠性概括的未来方向。(PsycInfo 数据库记录(c)2020 APA,保留所有权利)。