Schimmelpfennig Robin, Spicer Rachel, White Cindel J M, Gervais Will, Norenzayan Ara, Heine Steven, Henrich Joseph, Muthukrishna Michael
Faculty of Business and Economics, University of Lausanne, Lausanne, Switzerland.
Department of Psychological and Behavioural Science, London School of Economics and Political Science, London, United Kingdom.
Commun Psychol. 2024 Oct 8;2(1):93. doi: 10.1038/s44271-024-00135-z.
The multi-site replication study, Many Labs 2, concluded that sample location and setting did not substantially affect the replicability of findings. Here, we examine theoretical and methodological considerations for a subset of the analyses, namely exploratory tests of heterogeneity in the replicability of studies between "WEIRD and less-WEIRD cultures". We conducted a review of literature citing the study, a re-examination of the existing cultural variability, a power stimulation for detecting cultural heterogeneity, and re-analyses of the original exploratory tests. Findings indicate little cultural variability and low power to detect cultural heterogeneity effects in the Many Labs 2 data, yet the literature review indicates the study is cited regarding the moderating role of culture. Our reanalysis of the data found that using different operationalizations of culture slightly increased effect sizes but did not substantially alter the conclusions of Many Labs 2. Future studies of cultural heterogeneity can be improved with theoretical consideration of which effects and which cultures are likely to show variation as well as a priori methodological planning for appropriate operationalizations of culture and sufficient power to detect effects.
多地点重复研究“众多实验室2”得出结论，样本地点和环境对研究结果的可重复性影响不大。在此，我们探讨了部分分析的理论和方法学考量，即对“西方、受过良好教育、工业化、富有、民主（WEIRD）文化与非WEIRD文化”之间研究可重复性的异质性进行探索性检验。我们对引用该研究的文献进行了综述，重新审视了现有的文化变异性，对检测文化异质性进行了功效模拟，并对原始探索性检验进行了重新分析。研究结果表明，在“众多实验室2”的数据中，文化变异性很小，检测文化异质性效应的功效较低，但文献综述表明该研究被引用是关于文化的调节作用。我们对数据的重新分析发现，使用不同的文化操作化方法会略微增加效应量，但并没有实质性改变“众多实验室2”的结论。未来关于文化异质性的研究可以通过理论考量来改进哪些效应以及哪些文化可能会表现出差异，以及对文化的适当操作化和检测效应的足够功效进行先验方法学规划。
