Wright Bryan D, Vo Nam, Nolan Johnny, Johnson Austin L, Braaten Tyler, Tritz Daniel, Vassar Matt
Oklahoma State University Center for Health Sciences, 1111 W 17th St, Tulsa, OK, 74107, USA.
Kansas City University of Medicine and Biosciences, Joplin, MO, USA.
Insights Imaging. 2020 May 11;11(1):65. doi: 10.1186/s13244-020-00870-x.
Given the central role of radiology in patient care, it is important that radiological research is grounded in reproducible science. It is unclear whether there is a lack of reproducibility or transparency in radiologic research.
To analyze published radiology literature for the presence or lack of key indicators of reproducibility.
This cross-sectional retrospective study was performed by conducting a search of the National Library of Medicine (NLM) for publications contained within journals in the field of radiology. Our inclusion criteria were being MEDLINE indexed, written in English, and published from January 1, 2014, to December 31, 2018. We randomly sampled 300 publications for this study. A pilot-tested Google form was used to record information from the publications regarding indicators of reproducibility. Following peer-review, we extracted data from an additional 200 publications in an attempt to reproduce our initial results. The additional 200 publications were selected from the list of initially randomized publications.
Our initial search returned 295,543 records, from which 300 were randomly selected for analysis. Of these 300 records, 294 met inclusion criteria and 6 did not. Among the empirical publications, 5.6% (11/195, [3.0-8.3]) contained a data availability statement, 0.51% (1/195) provided clear documented raw data, 12.0% (23/191, [8.4-15.7]) provided a materials availability statement, 0% provided analysis scripts, 4.1% (8/195, [1.9-6.3]) provided a pre-registration statement, 2.1% (4/195, [0.4-3.7]) provided a protocol statement, and 3.6% (7/195, [1.5-5.7]) were pre-registered. The validation study of the 5 key indicators of reproducibility-availability of data, materials, protocols, analysis scripts, and pre-registration-resulted in 2 indicators (availability of protocols and analysis scripts) being reproduced, as they fell within the 95% confidence intervals for the proportions from the original sample. However, materials' availability and pre-registration proportions from the validation sample were lower than what was found in the original sample.
Our findings demonstrate key indicators of reproducibility are missing in the field of radiology. Thus, the ability to reproduce studies contained in radiology publications may be problematic and may have potential clinical implications.
鉴于放射学在患者护理中的核心作用，放射学研究基于可重复的科学至关重要。目前尚不清楚放射学研究是否缺乏可重复性或透明度。
分析已发表的放射学文献中是否存在可重复性的关键指标。
本横断面回顾性研究通过在美国国立医学图书馆（NLM）搜索放射学领域期刊中包含的出版物来进行。我们的纳入标准是被MEDLINE索引、用英文撰写且于2014年1月1日至2018年12月31日发表。本研究随机抽取了300篇出版物。使用经过预测试的谷歌表单记录出版物中有关可重复性指标的信息。经过同行评审后，我们从另外200篇出版物中提取数据，试图重现我们的初始结果。这另外200篇出版物是从最初随机抽取的出版物列表中选取的。
我们的初始搜索返回了295,543条记录，从中随机选择300条进行分析。在这300条记录中，294条符合纳入标准，6条不符合。在实证性出版物中，5.6%（11/195，[3.0 - 8.3]）包含数据可用性声明，0.51%（1/195）提供了清晰记录的原始数据，12.0%（23/191，[8.4 - 15.7]）提供了材料可用性声明，0%提供了分析脚本，4.1%（8/195，[1.9 - 6.3]）提供了预注册声明，2.1%（4/195，[0.4 - 3.7]）提供了方案声明，3.6%（7/195，[1.5 - 5.7]）进行了预注册。对数据、材料、方案、分析脚本和预注册这5个可重复性关键指标的验证研究结果显示，有2个指标（方案和分析脚本的可用性）得以重现，因为它们落在了原始样本比例的95%置信区间内。然而，验证样本中材料可用性和预注册比例低于原始样本中的比例。
我们的研究结果表明，放射学领域缺少可重复性的关键指标。因此，重现放射学出版物中所包含研究的能力可能存在问题，并且可能具有潜在的临床意义。
