Faculty of Psychology and Educational Sciences & ITEC, imec research group at KU Leuven, KU Leuven, University of Leuven, Leuven, Belgium.
Canadian Institute for Public Safety Research and Treatment (CIPSRT), University of Regina, Regina, Canada.
Behav Res Methods. 2020 Oct;52(5):2008-2019. doi: 10.3758/s13428-020-01380-w.
The focus of the current study is on handling the dependence among multiple regression coefficients representing the treatment effects when meta-analyzing data from single-case experimental studies. We compare the results when applying three different multilevel meta-analytic models (i.e., a univariate multilevel model avoiding the dependence, a multivariate multilevel model ignoring covariance at higher levels, and a multivariate multilevel model modeling the existing covariance) to deal with the dependent effect sizes. The results indicate better estimates of the overall treatment effects and variance components when a multivariate multilevel model is applied, independent of modeling or ignoring the existing covariance. These findings confirm the robustness of multilevel modeling to misspecifying the existing covariance at the case and study level in terms of estimating the overall treatment effects and variance components. The results also show that the overall treatment effect estimates are unbiased regardless of the underlying model, but the between-case and between-study variance components are biased in certain conditions. In addition, the between-study variance estimates are particularly biased when the number of studies is smaller than 40 (i.e., 10 or 20) and the true value of the between-case variance is relatively large (i.e., 8). The observed bias is larger for the between-case variance estimates compared to the between-study variance estimates when the true between-case variance is relatively small (i.e., 0.5).
本研究的重点是在对来自单病例实验研究的数据进行荟萃分析时处理代表治疗效果的多个回归系数之间的依存关系。我们比较了应用三种不同的多层次荟萃分析模型（即避免依存关系的单变量多层次模型、忽略较高层次协方差的多变量多层次模型以及对现有协方差进行建模的多变量多层次模型）处理依存效应大小的结果。结果表明，无论是否对现有协方差进行建模或忽略，应用多变量多层次模型都可以更好地估计总体治疗效果和方差分量。这些发现证实了多层次建模在估计总体治疗效果和方差分量方面，即使在病例和研究水平上指定现有协方差存在错误时，也具有稳健性。结果还表明，无论基础模型如何，总体治疗效果估计都是无偏的，但在某些条件下，病例间和研究间方差分量是有偏的。此外，当研究数量小于 40 时（即 10 或 20），并且病例间方差的真实值相对较大（即 8）时，研究间方差估计值特别有偏差。与研究间方差估计值相比，当病例间方差的真实值相对较小时（即 0.5），病例间方差估计值的观察偏差更大。
