Child Mind Institute, New York, NY, USA.
Masonic Institute for the Developing Brain, University of Minnesota, Minneapolis, MN, USA.
Nat Hum Behav. 2024 Oct;8(10):2003-2017. doi: 10.1038/s41562-024-01942-4. Epub 2024 Aug 5.
When fields lack consensus standard methods and accessible ground truths, reproducibility can be more of an ideal than a reality. Such has been the case for functional neuroimaging, where there exists a sprawling space of tools and processing pipelines. We provide a critical evaluation of the impact of differences across five independently developed minimal preprocessing pipelines for functional magnetic resonance imaging. We show that, even when handling identical data, interpipeline agreement was only moderate, critically shedding light on a factor that limits cross-study reproducibility. We show that low interpipeline agreement can go unrecognized until the reliability of the underlying data is high, which is increasingly the case as the field progresses. Crucially we show that, when interpipeline agreement is compromised, so too is the consistency of insights from brain-wide association studies. We highlight the importance of comparing analytic configurations, because both widely discussed and commonly overlooked decisions can lead to marked variation.
当领域缺乏共识的标准方法和可访问的地面真相时，可重复性更像是一种理想，而不是现实。功能神经影像学就是这种情况，其中存在着广泛的工具和处理管道空间。我们对五个独立开发的功能磁共振成像最小预处理管道的差异对其影响进行了批判性评估。我们表明，即使处理相同的数据，管道之间的一致性也只是中等，这严重地揭示了限制跨研究可重复性的一个因素。我们表明，直到基础数据的可靠性很高时，低管道一致性才会被忽视，而随着该领域的发展，这种情况越来越普遍。至关重要的是，我们表明，当管道之间的一致性受到损害时，脑关联研究的结果也不一致。我们强调了比较分析配置的重要性，因为广泛讨论和通常被忽视的决策都可能导致明显的变化。
