Hwang Jundong, Lustig Niv, Jung Minyoung, Lee Jong-Hwan
Department of Brain and Cognitive Engineering, Korea University, Seoul, South Korea.
Heliyon. 2023 Jul 16;9(7):e18086. doi: 10.1016/j.heliyon.2023.e18086. eCollection 2023 Jul.
Deep neural networks (DNNs) have been adopted widely as classifiers for functional magnetic resonance imaging (fMRI) data, advancing beyond traditional machine learning models. Consequently, transfer learning of the pre-trained DNN becomes crucial to enhance DNN classification performance, specifically by alleviating an overfitting issue that occurs when a substantial number of DNN parameters are fitted to a relatively small number of fMRI samples. In this study, we first systematically compared the two most popularly used, unsupervised pretraining models for resting-state fMRI (rfMRI) volume data to pre-train the DNNs, namely autoencoder (AE) and restricted Boltzmann machine (RBM). The group in-brain mask used when training AE and RBM displayed a sizable overlap ratio with Yeo's seven functional brain networks (FNs). The parcellated FNs obtained from the RBM were fine-grained compared to those from the AE. The pre-trained AE and RBM served as the weight parameters of the first of the two hidden DNN layers, and the DNN fulfilled the task classifier role for fMRI (tfMRI) data in the Human Connectome Project (HCP). We tested two transfer learning schemes: (1) fixing and (2) fine-tuning the DNN's pre-trained AE or RBM weights. The DNN with transfer learning was compared to a baseline DNN, trained using random initial weights. Overall, DNN classification performance from the transfer learning proved superior when the pre-trained RBM weights were fixed and when the pre-trained AE weights were fine-tuned (average error rates: 14.8% for fixed RBM, 15.1% fine-tuned AE, and 15.5% for the baseline model) compared to the alternative scenarios of DNN transfer learning schemes. Moreover, the optimal transfer learning scheme between the fixed RBM and fine-tuned AE varied according to seven task conditions in the HCP. Nonetheless, the computational load reduced substantially for the fixed-weight-based transfer learning compared to the fine-tuning-based transfer learning (e.g., the number of weight parameters for the fixed-weight-based DNN model reduced to 1.9% compared with a baseline/fine-tuned DNN model). Our findings suggest that weight initialization at the DNN's first layer using RBM-based pre-trained weights provides the most promising approach when the whole-brain fMRI volume supports associated task classification. We believe that our proposed scheme could be applied to a variety of task conditions to improve their classification performance and to utilize computational resources efficiently using our AE/RBM-based pre-trained weights compared to random initial weights for DNN training.
深度神经网络（DNN）已被广泛用作功能磁共振成像（fMRI）数据的分类器，超越了传统的机器学习模型。因此，预训练DNN的迁移学习对于提高DNN分类性能至关重要，特别是通过缓解当大量DNN参数拟合到相对较少的fMRI样本时出现的过拟合问题。在本研究中，我们首先系统地比较了两种最常用的用于静息态fMRI（rfMRI）体积数据的无监督预训练模型，以预训练DNN，即自动编码器（AE）和受限玻尔兹曼机（RBM）。训练AE和RBM时使用的组内脑掩码与Yeo的七个功能性脑网络（FN）显示出相当大的重叠率。与从AE获得的那些相比，从RBM获得的分割后的FN更精细。预训练的AE和RBM作为两个隐藏DNN层中第一个的权重参数，并且DNN在人类连接组计划（HCP）中充当fMRI（tfMRI）数据的任务分类器。我们测试了两种迁移学习方案：（1）固定和（2）微调DNN的预训练AE或RBM权重。将具有迁移学习的DNN与使用随机初始权重训练的基线DNN进行比较。总体而言，与DNN迁移学习方案的其他情况相比，当预训练的RBM权重固定且预训练的AE权重微调时，迁移学习的DNN分类性能证明更优（平均错误率：固定RBM为14.8%，微调AE为15.1%，基线模型为15.5%）。此外，固定RBM和微调AE之间的最佳迁移学习方案根据HCP中的七个任务条件而有所不同。尽管如此，与基于微调的迁移学习相比，基于固定权重的迁移学习的计算负荷大幅降低（例如，基于固定权重的DNN模型的权重参数数量与基线/微调DNN模型相比减少到1.9%）。我们的研究结果表明，当全脑fMRI体积支持相关任务分类时，使用基于RBM的预训练权重在DNN的第一层进行权重初始化提供了最有前景的方法。我们相信，与用于DNN训练的随机初始权重相比，我们提出的方案可以应用于各种任务条件，以提高它们的分类性能并有效利用计算资源，使用我们基于AE/RBM的预训练权重。
