Big Data Institute, Li Ka Shing Centre for Health Information and Discovery, Nuffield Department of Population Health, University of Oxford, Oxford, UK.
Inria, Univ Rennes, CNRS, Inserm, IRISA UMR 6074, Empenn ERL U 1228, Rennes, France.
Hum Brain Mapp. 2019 Aug 1;40(11):3362-3384. doi: 10.1002/hbm.24603. Epub 2019 May 2.
A wealth of analysis tools are available to fMRI researchers in order to extract patterns of task variation and, ultimately, understand cognitive function. However, this "methodological plurality" comes with a drawback. While conceptually similar, two different analysis pipelines applied on the same dataset may not produce the same scientific results. Differences in methods, implementations across software, and even operating systems or software versions all contribute to this variability. Consequently, attention in the field has recently been directed to reproducibility and data sharing. In this work, our goal is to understand how choice of software package impacts on analysis results. We use publicly shared data from three published task fMRI neuroimaging studies, reanalyzing each study using the three main neuroimaging software packages, AFNI, FSL, and SPM, using parametric and nonparametric inference. We obtain all information on how to process, analyse, and model each dataset from the publications. We make quantitative and qualitative comparisons between our replications to gauge the scale of variability in our results and assess the fundamental differences between each software package. Qualitatively we find similarities between packages, backed up by Neurosynth association analyses that correlate similar words and phrases to all three software package's unthresholded results for each of the studies we reanalyse. However, we also discover marked differences, such as Dice similarity coefficients ranging from 0.000 to 0.684 in comparisons of thresholded statistic maps between software. We discuss the challenges involved in trying to reanalyse the published studies, and highlight our efforts to make this research reproducible.
有大量的分析工具可供 fMRI 研究人员使用，以便提取任务变化的模式，最终理解认知功能。然而，这种“方法多样性”也有一个缺点。虽然概念上相似，但应用于同一数据集的两个不同分析管道可能不会产生相同的科学结果。方法上的差异、软件之间的实现差异，甚至操作系统或软件版本差异都会导致这种可变性。因此，最近该领域的注意力集中在可重复性和数据共享上。在这项工作中，我们的目标是了解软件包的选择如何影响分析结果。我们使用来自三个已发表的任务 fMRI 神经影像学研究的公开共享数据，使用 AFNI、FSL 和 SPM 这三个主要的神经影像学软件包重新分析每个研究，使用参数和非参数推断。我们从出版物中获取处理、分析和建模每个数据集的所有信息。我们对我们的复制进行定量和定性比较，以衡量我们结果的可变性规模，并评估每个软件包之间的基本差异。从定性上看，我们发现软件包之间存在相似之处，这得到了神经综合关联分析的支持，该分析将相似的单词和短语与我们重新分析的每项研究的所有三个软件包的未阈值结果相关联。然而，我们也发现了明显的差异，例如在软件之间阈值统计地图的比较中，Dice 相似系数从 0.000 到 0.684 不等。我们讨论了尝试重新分析已发表研究所涉及的挑战，并强调了我们为使这项研究具有可重复性而做出的努力。
