Aktı Şeymanur, Kamar Doğay, Özlü Özgür Anıl, Soydemir Ihsan, Akcan Muhammet, Kul Abdullah, Rekik Islem
Faculty of Computer and Informatics, Istanbul Technical University, Turkey.
Faculty of Computer and Informatics, Istanbul Technical University, Turkey.
J Neurosci Methods. 2022 Feb 15;368:109475. doi: 10.1016/j.jneumeth.2022.109475. Epub 2022 Jan 4.
Predicting the evolution of the brain network, also called connectome, by foreseeing changes in the connectivity weights linking pairs of anatomical regions makes it possible to spot connectivity-related neurological disorders in earlier stages and detect the development of potential connectomic anomalies. Remarkably, such a challenging prediction problem remains least explored in the predictive connectomics literature. It is a known fact that machine learning (ML) methods have proven their predictive abilities in a wide variety of computer vision problems. However, ML techniques specifically tailored for the prediction of brain connectivity evolution trajectory from a single timepoint are almost absent.
To fill this gap, we organized a Kaggle competition where 20 competing teams designed advanced machine learning pipelines for predicting the brain connectivity evolution from a single timepoint. The teams developed their ML pipelines with combination of data pre-processing, dimensionality reduction and learning methods. Each ML framework inputs a baseline brain connectivity matrix observed at baseline timepoint t and outputs the brain connectivity map at a follow-up timepoint t. The longitudinal OASIS-2 dataset was used for model training and evaluation. Both random data split and 5-fold cross-validation strategies were used for ranking and evaluating the generalizability and scalability of each competing ML pipeline.
Utilizing an inclusive approach, we ranked the methods based on two complementary evaluation metrics (mean absolute error (MAE) and Pearson Correlation Coefficient (PCC)) and their performances using different training and testing data perturbation strategies (single random split and cross-validation). The final rank was calculated using the rank product for each competing team across all evaluation measures and validation strategies. Furthermore, we added statistical significance values to each proposed pipeline.
In support of open science, the developed 20 ML pipelines along with the connectomic dataset are made available on GitHub (https://github.com/basiralab/Kaggle-BrainNetPrediction-Toolbox). The outcomes of this competition are anticipated to lead the further development of predictive models that can foresee the evolution of the brain connectivity over time, as well as other types of networks (e.g., genetic networks).
通过预测连接解剖区域对的连接权重变化来预测脑网络（也称为连接组）的演变，有助于在早期阶段发现与连接性相关的神经系统疾病，并检测潜在连接组异常的发展。值得注意的是，在预测性连接组学文献中，这个具有挑战性的预测问题仍未得到充分探索。机器学习（ML）方法在各种计算机视觉问题中已证明其预测能力，这是一个众所周知的事实。然而，专门为从单个时间点预测脑连接演变轨迹而量身定制的ML技术几乎不存在。
为了填补这一空白，我们组织了一场Kaggle竞赛，20个参赛团队设计了先进的机器学习管道，用于从单个时间点预测脑连接演变。这些团队通过数据预处理、降维和学习方法的组合来开发他们的ML管道。每个ML框架输入在基线时间点t观察到的基线脑连接矩阵，并输出随访时间点t的脑连接图。纵向OASIS - 2数据集用于模型训练和评估。随机数据分割和5折交叉验证策略均用于对每个竞争的ML管道的泛化性和可扩展性进行排名和评估。
我们采用一种综合方法，基于两个互补的评估指标（平均绝对误差（MAE）和皮尔逊相关系数（PCC））以及它们在不同训练和测试数据扰动策略（单次随机分割和交叉验证）下的性能对方法进行排名。最终排名是使用每个竞争团队在所有评估措施和验证策略下的排名乘积来计算的。此外，我们为每个提出的管道添加了统计显著性值。
为支持开放科学，已在GitHub（https://github.com/basiralab/Kaggle - BrainNetPrediction - Toolbox）上提供了开发的20个ML管道以及连接组数据集。预计本次竞赛的结果将引领预测模型的进一步发展，这些模型能够预见脑连接随时间的演变以及其他类型的网络（例如，遗传网络）。
