Faculty of Psychology and Educational Sciences, KU Leuven, University of Leuven, IICK Building, Box 1.33, Etienne Sabelaan 51, 8500, Kortrijk, Belgium.
Imec-ITEC, KU Leuven, University of Leuven, Leuven, Belgium.
Behav Res Methods. 2019 Jun;51(3):1286-1304. doi: 10.3758/s13428-018-1063-2.
It is common for the primary studies in meta-analyses to report multiple effect sizes, generating dependence among them. Hierarchical three-level models have been proposed as a means to deal with this dependency. Sometimes, however, dependency may be due to multiple random factors, and random factors are not necessarily nested, but rather may be crossed. For instance, effect sizes may belong to different studies, and, at the same time, effect sizes might represent the effects on different outcomes. Cross-classified random-effects models (CCREMs) can be used to model this nonhierarchical dependent structure. In this article, we explore by means of a simulation study the performance of CCREMs in comparison with the use of other meta-analytic models and estimation procedures, including the use of three- and two-level models and robust variance estimation. We also evaluated the performance of CCREMs when the underlying data were generated using a multivariate model. The results indicated that, whereas the quality of fixed-effect estimates is unaffected by any misspecification in the model, the standard error estimates of the mean effect size and of the moderator variables' effects, as well as the variance component estimates, are biased under some conditions. Applying CCREMs led to unbiased fixed-effect and variance component estimates, outperforming the other models. Even when a CCREM was not used to generate the data, applying the CCREM yielded sound parameter estimates and inferences.
在荟萃分析中，主要研究通常会报告多个效应大小，从而产生它们之间的依赖性。分层三级模型已被提出作为处理这种依赖性的一种方法。然而，有时依赖性可能是由于多个随机因素引起的，并且随机因素不一定嵌套，而是可能交叉。例如，效应大小可能属于不同的研究，同时，效应大小可能代表对不同结果的影响。交叉分类随机效应模型（CCREM）可用于对这种非层次依赖结构进行建模。在本文中，我们通过模拟研究来探讨 CCREM 与其他荟萃分析模型和估计程序（包括使用三水平和两水平模型以及稳健方差估计）的性能比较。我们还评估了当使用多元模型生成基础数据时 CCREM 的性能。结果表明，虽然固定效应估计的质量不受模型任何不恰当指定的影响，但在某些条件下，平均效应大小和调节变量效应的标准误差估计以及方差分量估计会有偏差。应用 CCREM 可以得到无偏的固定效应和方差分量估计，表现优于其他模型。即使 CCREM 没有用于生成数据，应用 CCREM 也可以得到可靠的参数估计和推断。
