Hardwicke Tom E, Wallach Joshua D, Kidwell Mallory C, Bendixen Theiss, Crüwell Sophia, Ioannidis John P A
Meta-Research Innovation Center Berlin (METRIC-B), QUEST Center for Transforming Biomedical Research, Berlin Institute of Health, Charité - Universitätsmedizin Berlin, Anna-Louisa-Karsch-Str.2, 10178 Berlin, Germany.
Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA.
R Soc Open Sci. 2020 Feb 19;7(2):190806. doi: 10.1098/rsos.190806. eCollection 2020 Feb.
Serious concerns about research quality have catalysed a number of reform initiatives intended to improve transparency and reproducibility and thus facilitate self-correction, increase efficiency and enhance research credibility. Meta-research has evaluated the merits of some individual initiatives; however, this may not capture broader trends reflecting the cumulative contribution of these efforts. In this study, we manually examined a random sample of 250 articles in order to estimate the prevalence of a range of transparency and reproducibility-related indicators in the social sciences literature published between 2014 and 2017. Few articles indicated availability of materials (16/151, 11% [95% confidence interval, 7% to 16%]), protocols (0/156, 0% [0% to 1%]), raw data (11/156, 7% [2% to 13%]) or analysis scripts (2/156, 1% [0% to 3%]), and no studies were pre-registered (0/156, 0% [0% to 1%]). Some articles explicitly disclosed funding sources (or lack of; 74/236, 31% [25% to 37%]) and some declared no conflicts of interest (36/236, 15% [11% to 20%]). Replication studies were rare (2/156, 1% [0% to 3%]). Few studies were included in evidence synthesis via systematic review (17/151, 11% [7% to 16%]) or meta-analysis (2/151, 1% [0% to 3%]). Less than half the articles were publicly available (101/250, 40% [34% to 47%]). Minimal adoption of transparency and reproducibility-related research practices could be undermining the credibility and efficiency of social science research. The present study establishes a baseline that can be revisited in the future to assess progress.
对研究质量的严重担忧催生了一系列旨在提高透明度和可重复性、从而促进自我修正、提高效率并增强研究可信度的改革举措。元研究评估了一些个别举措的优点；然而，这可能无法捕捉到反映这些努力累积贡献的更广泛趋势。在本研究中，我们手动检查了250篇文章的随机样本，以估计2014年至2017年发表的社会科学文献中一系列与透明度和可重复性相关指标的流行情况。很少有文章表明材料（16/151，11%[95%置信区间，7%至16%]）、方案（0/156，0%[0%至1%]）、原始数据（11/156，7%[2%至13%]）或分析脚本（2/156，1%[0%至3%]）可用，且没有研究进行预注册（0/156，0%[0%至1%]）。一些文章明确披露了资金来源（或缺乏资金来源；74/236，31%[25%至37%]），一些声明没有利益冲突（36/236，15%[11%至20%]）。复制研究很少见（2/156，1%[0%至3%]）。很少有研究通过系统评价（17/151，11%[7%至16%]）或元分析（2/151，1%[0%至3%]）纳入证据综合。不到一半的文章是公开可用的（101/250，40%[34%至47%]）。对与透明度和可重复性相关的研究实践的最低限度采用可能正在损害社会科学研究的可信度和效率。本研究建立了一个基线，未来可以重新审视以评估进展情况。
