Germani Elodie, Bhagwat Nikhil, Dugré Mathieu, Gau Rémi, Montillo Albert A, Nguyen Kevin P, Sokolowski Andrzej, Sharp Madeleine, Poline Jean-Baptiste, Glatard Tristan
Univ Rennes, Inria, CNRS, Inserm, Rennes, France.
Department of Neurology and Neurosurgery, McGill University, Montreal, Canada.
PLoS One. 2025 Feb 21;20(2):e0317566. doi: 10.1371/journal.pone.0317566. eCollection 2025.
Parkinson's disease (PD) is a common neurodegenerative disorder with a poorly understood physiopathology and no established biomarkers for the diagnosis of early stages and for prediction of disease progression. Several neuroimaging biomarkers have been studied recently, but these are susceptible to several sources of variability related for instance to cohort selection or image analysis. In this context, an evaluation of the robustness of such biomarkers to variations in the data processing workflow is essential. This study is part of a larger project investigating the replicability of potential neuroimaging biomarkers of PD. Here, we attempt to fully reproduce (reimplementing the experiments with the same methods, including data collection from the same database) and replicate (different data and/or method) the models described in (Nguyen et al., 2021) to predict individual's PD current state and progression using demographic, clinical and neuroimaging features (fALFF and ReHo extracted from resting-state fMRI). We use the Parkinson's Progression Markers Initiative dataset (PPMI, ppmi-info.org), as in (Nguyen et al., 2021) and aim to reproduce the original cohort, imaging features and machine learning models as closely as possible using the information available in the paper and the code. We also investigated methodological variations in cohort selection, feature extraction pipelines and sets of input features. Different criteria were used to evaluate the reproduction attempt and compare the results with the original ones. Notably, we obtained significantly better than chance performance using the analysis pipeline closest to that in the original study (R2 > 0), which is consistent with its findings. In addition, we performed a partial reproduction using derived data provided by the authors of the original study, and we obtained results that were close to the original ones. The challenges encountered while attempting to reproduce (fully and partially) and replicating the original work are likely explained by the complexity of neuroimaging studies, in particular in clinical settings. We provide recommendations to further facilitate the reproducibility of such studies in the future.
帕金森病(PD)是一种常见的神经退行性疾病,其生理病理学尚不清楚,且尚无用于早期诊断和疾病进展预测的既定生物标志物。最近对几种神经影像学生物标志物进行了研究,但这些标志物容易受到多种变异性来源的影响,例如与队列选择或图像分析有关。在这种情况下,评估此类生物标志物对数据处理工作流程变化的稳健性至关重要。本研究是一个更大项目的一部分,该项目旨在调查PD潜在神经影像学生物标志物的可重复性。在此,我们尝试完全重现(使用相同方法重新进行实验,包括从同一数据库收集数据)并复制(不同的数据和/或方法)(Nguyen等人,2021年)中描述的模型,以使用人口统计学、临床和神经影像学特征(从静息态功能磁共振成像中提取的fALFF和ReHo)来预测个体的PD当前状态和进展。我们使用帕金森病进展标志物倡议数据集(PPMI,ppmi - info.org),如同(Nguyen等人,2021年)中那样,并旨在利用论文中提供的信息和代码尽可能紧密地重现原始队列、影像特征和机器学习模型。我们还研究了队列选择、特征提取管道和输入特征集方面的方法学差异。使用不同标准来评估重现尝试并将结果与原始结果进行比较。值得注意的是,使用与原始研究最接近的分析管道,我们获得了显著优于随机水平的性能(R2 > 0),这与该研究的结果一致。此外,我们使用原始研究作者提供的派生数据进行了部分重现,并且获得了与原始结果相近的结果。在尝试完全和部分重现以及复制原始工作时遇到的挑战,很可能是由神经影像学研究的复杂性所解释的,尤其是在临床环境中。我们提供了一些建议,以在未来进一步促进此类研究的可重复性。
Parkinsonism Relat Disord. 2021-4
J Open Source Softw. 2021
Patterns (N Y). 2023-8-4
J Neural Transm (Vienna). 2023-6
Biol Psychiatry Cogn Neurosci Neuroimaging. 2023-8
Front Aging Neurosci. 2022-1-25
PLoS Comput Biol. 2022-1-6
Elife. 2021-10-18
JAMA Neurol. 2021-10-1
Lancet. 2021-6-12