Amsterdam Center for Language and Communication, University of Amsterdam, Spuistraat 134, Amsterdam, 1012 VB, The Netherlands.
Department of Methodology & Statistics, Utrecht University, Utrecht, The Netherlands.
Behav Res Methods. 2024 Apr;56(4):4085-4102. doi: 10.3758/s13428-024-02350-2. Epub 2024 Mar 26.
Synthesizing results across multiple studies is a popular way to increase the robustness of scientific findings. The most well-known method for doing this is meta-analysis. However, because meta-analysis requires conceptually comparable effect sizes with the same statistical form, meta-analysis may not be possible when studies are highly diverse in terms of their research design, participant characteristics, or operationalization of key variables. In these situations, Bayesian evidence synthesis may constitute a flexible and feasible alternative, as this method combines studies at the hypothesis level rather than at the level of the effect size. This method therefore poses less constraints on the studies to be combined. In this study, we introduce Bayesian evidence synthesis and show through simulations when this method diverges from what would be expected in a meta-analysis to help researchers correctly interpret the synthesis results. As an empirical demonstration, we also apply Bayesian evidence synthesis to a published meta-analysis on statistical learning in people with and without developmental language disorder. We highlight the strengths and weaknesses of the proposed method and offer suggestions for future research.
综合多项研究的结果是提高科学发现稳健性的一种常用方法。最著名的方法是元分析。然而,由于元分析需要具有相同统计形式的概念上可比的效应量,因此当研究在研究设计、参与者特征或关键变量的操作化方面高度多样化时,元分析可能不可行。在这些情况下,贝叶斯证据综合可能是一种灵活且可行的替代方法,因为这种方法在假设层面而不是在效应量层面上综合研究。因此,这种方法对要综合的研究施加的限制较少。在这项研究中,我们介绍了贝叶斯证据综合,并通过模拟展示了当这种方法与元分析中预期的结果不一致时,如何帮助研究人员正确解释综合结果。作为实证演示,我们还将贝叶斯证据综合应用于一项关于有和没有发育性语言障碍的人在统计学习方面的已发表的元分析。我们强调了所提出方法的优缺点,并为未来的研究提供了建议。