Faculty of Psychology and Educational Sciences, KU Leuven, Leuven, Belgium.
Behav Res Methods. 2020 Apr;52(2):654-666. doi: 10.3758/s13428-019-01266-6.
Multilevel models (MLMs) have been proposed in single-case research, to synthesize data from a group of cases in a multiple-baseline design (MBD). A limitation of this approach is that MLMs require several statistical assumptions that are often violated in single-case research. In this article we propose a solution to this limitation by presenting a randomization test (RT) wrapper for MLMs that offers a nonparametric way to evaluate treatment effects, without making distributional assumptions or an assumption of random sampling. We present the rationale underlying the proposed technique and validate its performance (with respect to Type I error rate and power) as compared to parametric statistical inference in MLMs, in the context of evaluating the average treatment effect across cases in an MBD. We performed a simulation study that manipulated the numbers of cases and of observations per case in a dataset, the data variability between cases, the distributional characteristics of the data, the level of autocorrelation, and the size of the treatment effect in the data. The results showed that the power of the RT wrapper is superior to the power of parametric tests based on F distributions for MBDs with fewer than five cases, and that the Type I error rate of the RT wrapper is controlled for bimodal data, whereas this is not the case for traditional MLMs.
多水平模型 (MLMs) 已在单案例研究中提出，以综合多个基线设计 (MBD) 中一组案例的数据。这种方法的一个限制是，MLMs 需要满足一些统计假设，而这些假设在单案例研究中经常被违反。在本文中，我们通过提出一个用于 MLMs 的随机化检验 (RT) 包装器来解决这个限制，该包装器提供了一种非参数方法来评估治疗效果，而无需做出分布假设或随机抽样假设。我们介绍了所提出技术的基本原理，并验证了其在评估 MBD 中案例间平均治疗效果方面的性能（相对于 MLMs 中的参数统计推断的错误率和功效）。我们进行了一项模拟研究，该研究在数据集内操纵了案例数量和每个案例的观察数量、案例之间的数据变异性、数据的分布特征、自相关程度以及数据中的治疗效果大小。结果表明，对于少于五个案例的 MBD，RT 包装器的功效优于基于 F 分布的参数检验的功效，并且 RT 包装器的 I 型错误率得到了控制对于双峰数据，而传统的 MLMs 则不是这样。
