Pollo Pietro, Lagisz Malgorzata, Macedo-Rego Renato C, Mizuno Ayumi, Yang Yefeng, Nakagawa Shinichi
Evolution & Ecology Research Centre, School of Biological, Earth & Environmental Sciences, University of New South Wales, Kensington, New South Wales, Australia.
Departamento de Biologia Geral, Universidade Federal de Viçosa, Viçosa, Minas Gerais, Brazil.
Proc Biol Sci. 2025 May;292(2047):20242782. doi: 10.1098/rspb.2024.2782. Epub 2025 May 21.
Meta-analyses are powerful synthesis tools that are popular in ecology and evolution owing to the rapidly growing literature of this field. Although the usefulness of meta-analyses depends on their reliability, such as the precision of individual and mean effect sizes, attempts to reproduce meta-analyses' results remain rare in ecology and evolution. Here, we assess the reliability of 41 meta-analyses on sexual signals by evaluating the reproducibility and replicability of their results. We attempted to: (i) reproduce meta-analyses' mean effect sizes using the datasets they provided; (ii) reproduce meta-analyses' effect sizes by re-extracting 5703 effect sizes from 246 primary studies they used as sources; (iii) assess the extent of relevant data missed by original meta-analyses; and (iv) replicate meta-analyses' mean effect sizes after incorporating re-extracted and relevant missing data. We found many discrepancies between meta-analyses' reported results and those generated by our analyses for all reproducibility and replicability attempts. Nonetheless, we argue that the meta-analyses we evaluated are largely reproducible and replicable because the differences we found were small in magnitude, leaving the original interpretation of these meta-analyses' results unchanged. Still, we highlight issues we observed in these meta-analyses that affected their reliability, providing recommendations to ameliorate them.
元分析是强大的综合工具,由于生态学和进化领域的文献迅速增长,它们在该领域很受欢迎。尽管元分析的有用性取决于其可靠性,如个体效应量和平均效应量的精度,但在生态学和进化领域,重现元分析结果的尝试仍然很少见。在这里,我们通过评估41项关于性信号的元分析结果的可重复性和可复制性来评估其可靠性。我们试图:(i) 使用它们提供的数据集重现元分析的平均效应量;(ii) 通过从它们用作来源的246项原始研究中重新提取5703个效应量来重现元分析的效应量;(iii) 评估原始元分析遗漏的相关数据的程度;以及(iv) 在纳入重新提取的和相关的缺失数据后复制元分析的平均效应量。对于所有的可重复性和可复制性尝试,我们发现元分析报告的结果与我们的分析产生的结果之间存在许多差异。尽管如此,我们认为我们评估的元分析在很大程度上是可重复和可复制的,因为我们发现的差异在量级上很小,这些元分析结果的原始解释保持不变。不过,我们强调了在这些元分析中观察到的影响其可靠性的问题,并提供了改进这些问题的建议。
Cochrane Database Syst Rev. 2025-5-7
Cochrane Database Syst Rev. 2022-6-15
Cochrane Database Syst Rev. 2025-6-13
Cochrane Database Syst Rev. 2025-6-4
Cochrane Database Syst Rev. 2025-3-10
Cochrane Database Syst Rev. 2025-6-19
Cochrane Database Syst Rev. 2025-4-14
Biol Rev Camb Philos Soc. 2024-12
Res Synth Methods. 2023-11
Behav Ecol. 2023-2-23
Nat Commun. 2022-10-7
Proc Biol Sci. 2022-6-8