Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Palo Alto, CA 94304;
Centre for Science and Technology Studies, Leiden University, 2333 AL Leiden, The Netherlands.
Proc Natl Acad Sci U S A. 2017 Apr 4;114(14):3714-3719. doi: 10.1073/pnas.1618569114. Epub 2017 Mar 20.
Numerous biases are believed to affect the scientific literature, but their actual prevalence across disciplines is unknown. To gain a comprehensive picture of the potential imprint of bias in science, we probed for the most commonly postulated bias-related patterns and risk factors, in a large random sample of meta-analyses taken from all disciplines. The magnitude of these biases varied widely across fields and was overall relatively small. However, we consistently observed a significant risk of small, early, and highly cited studies to overestimate effects and of studies not published in peer-reviewed journals to underestimate them. We also found at least partial confirmation of previous evidence suggesting that US studies and early studies might report more extreme effects, although these effects were smaller and more heterogeneously distributed across meta-analyses and disciplines. Authors publishing at high rates and receiving many citations were, overall, not at greater risk of bias. However, effect sizes were likely to be overestimated by early-career researchers, those working in small or long-distance collaborations, and those responsible for scientific misconduct, supporting hypotheses that connect bias to situational factors, lack of mutual control, and individual integrity. Some of these patterns and risk factors might have modestly increased in intensity over time, particularly in the social sciences. Our findings suggest that, besides one being routinely cautious that published small, highly-cited, and earlier studies may yield inflated results, the feasibility and costs of interventions to attenuate biases in the literature might need to be discussed on a discipline-specific and topic-specific basis.
人们认为许多偏见会影响科学文献，但它们在不同学科中的实际普遍性尚不清楚。为了全面了解科学中潜在偏见的影响，我们在来自所有学科的大量随机元分析样本中，探究了最常见的假设偏见相关模式和风险因素。这些偏见在不同领域的幅度差异很大，总体上相对较小。然而，我们一致观察到，小、早期和高引用的研究有高估效应的显著风险，而未在同行评议期刊上发表的研究有低估效应的风险。我们还至少部分证实了先前的证据，表明美国的研究和早期的研究可能报告更极端的效应，尽管这些效应在元分析和学科中较小且更不均匀分布。总体而言，高发表率和高引用率的作者并不存在更大的偏见风险。然而，早期职业生涯的研究人员、在小型或远距离合作的研究人员以及负责科学不端行为的研究人员，他们的效应大小可能被高估，这支持了将偏见与情境因素、缺乏相互控制和个人诚信联系起来的假设。这些模式和风险因素中的一些可能随着时间的推移略有增加，尤其是在社会科学中。我们的研究结果表明，除了要例行谨慎地认为已发表的小、高引用和早期研究可能产生夸大的结果外，还需要在特定学科和特定主题的基础上，讨论减轻文献偏见的可行性和成本。
