Paul Sabatier University, Dental Faculty, Department of Anatomical Sciences and Radiology, Toulouse University Hospital (CHU de Toulouse), 31062 Toulouse Cedex 9 and STROMALab, Université de Toulouse, CNRS ERL 5311, EFS, ENVT, Inserm, UPS, 31432 Toulouse Cedex 4, France.
Paul Sabatier University, Dental Faculty, Department of Epidemiology and Public Health, Toulouse University Hospital (CHU de Toulouse), 31062 Toulouse Cedex 9, France and Division of Oral Health and Society, Faculty of dentistry, McGill University, Montreal, Quebec, QC H3A 0G4, Canada.
Gigascience. 2018 Jan 1;7(1):1-10. doi: 10.1093/gigascience/gix121.
In medicine, effect sizes (ESs) allow the effects of independent variables (including risk/protective factors or treatment interventions) on dependent variables (e.g., health outcomes) to be quantified. Given that many public health decisions and health care policies are based on ES estimates, it is important to assess how ESs are used in the biomedical literature and to investigate potential trends in their reporting over time.
Through a big data approach, the text mining process automatically extracted 814 120 ESs from 13 322 754 PubMed abstracts. Eligible ESs were risk ratio, odds ratio, and hazard ratio, along with their confidence intervals. Here we show a remarkable decrease of ES values in PubMed abstracts between 1990 and 2015 while, concomitantly, results become more often statistically significant. Medians of ES values have decreased over time for both "risk" and "protective" values. This trend was found in nearly all fields of biomedical research, with the most marked downward tendency in genetics. Over the same period, the proportion of statistically significant ESs increased regularly: among the abstracts with at least 1 ES, 74% were statistically significant in 1990-1995, vs 85% in 2010-2015.
whereas decreasing ESs could be an intrinsic evolution in biomedical research, the concomitant increase of statistically significant results is more intriguing. Although it is likely that growing sample sizes in biomedical research could explain these results, another explanation may lie in the "publish or perish" context of scientific research, with the probability of a growing orientation toward sensationalism in research reports. Important provisions must be made to improve the credibility of biomedical research and limit waste of resources.
在医学领域，效应大小（ES）可用于量化自变量（包括风险/保护因素或治疗干预）对因变量（例如健康结果）的影响。鉴于许多公共卫生决策和医疗保健政策都是基于 ES 估计值制定的，因此评估 ES 在生物医学文献中的使用情况并研究其报告随时间变化的潜在趋势非常重要。
通过大数据方法，文本挖掘过程自动从 13322754 篇 PubMed 摘要中提取了 814120 个 ES。合格的 ES 是风险比、优势比和风险比及其置信区间。在这里，我们展示了 1990 年至 2015 年期间 PubMed 摘要中 ES 值的显著下降，同时结果变得更加具有统计学意义。风险和保护值的 ES 值中位数随时间推移而降低。这种趋势在几乎所有生物医学研究领域都存在，在遗传学中下降趋势最为明显。在此期间，具有统计学意义的 ES 比例定期增加：在至少有 1 个 ES 的摘要中，1990-1995 年有 74%具有统计学意义，而 2010-2015 年有 85%。
虽然 ES 值的降低可能是生物医学研究的内在演变，但同时统计学上显著结果的增加更令人关注。尽管生物医学研究中样本量的增加可能解释了这些结果，但另一种解释可能在于科学研究的“发表或灭亡”背景下，研究报告中对轰动效应的倾向越来越大。必须采取重要措施来提高生物医学研究的可信度并限制资源浪费。
