School of Physics and Information Technology, Shaanxi Normal University, Xi'an, China; Center for Brain and Brain-Inspired Computing Research, Department of Computer Science, Northwestern Polytechnical University, Xi'an, China.
Advanced Medical Computing and Analysis, Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA.
Comput Methods Programs Biomed. 2022 Aug;223:106979. doi: 10.1016/j.cmpb.2022.106979. Epub 2022 Jun 27.
To understand brain cognition and disorders, modeling the mapping between mind and brain has been of great interest to the neuroscience community. The key is the brain representation, including functional brain networks (FBN) and their corresponding temporal features. Recently, it has been proven that deep learning models have superb representation power on functional magnetic resonance imaging (fMRI) over traditional machine learning methods. However, due to the lack of high-quality data and labels, deep learning models tend to suffer from overfitting in the training process.
In this work, we applied a recurrent Wasserstein generative adversarial net (RWGAN) to learn brain representation from volumetric fMRI data. Generative adversarial net (GAN) is widely used in natural image generation and is able to capture the distribution of the input data, which enables the extraction of generalized features from fMRI and thus relieves the overfitting issue. The recurrent layers in RWGAN are designed to better model the local temporal features of the fMRI time series. The discriminator of RWGAN works as a deep feature extractor. With LASSO regression, the RWGAN model can decompose the fMRI data into temporal features and spatial features (FBNs). Furthermore, the generator of RWGAN can generate high-quality new data for fMRI augmentation.
The experimental results on seven tasks from the HCP dataset showed that the RWGAN can learn meaningful and interpretable temporal features and FBNs, compared to HCP task designs and general linear model (GLM) derived networks. Besides, the results on different training datasets showed that the RWGAN performed better on small datasets than other deep learning models. Moreover, we used the generator of RWGAN to yield fake subjects. The result showed that the fake data can also be used to learn meaningful representation compared to those learned from real data.
To our best knowledge, this work is among the earliest attempts of applying generative deep learning for modeling fMRI data. The proposed RWGAN offers a novel methodology for learning brain representation from fMRI, and it can generate high-quality fake data for the potential use of fMRI data augmentation.
为了理解大脑认知和障碍，对大脑与思维之间的映射进行建模一直是神经科学界的研究热点。关键在于大脑的表示，包括功能脑网络（FBN）及其对应的时间特征。最近，研究已经证明，与传统机器学习方法相比，深度学习模型在功能磁共振成像（fMRI）上具有卓越的表示能力。然而，由于缺乏高质量的数据和标签，深度学习模型在训练过程中容易出现过拟合。
在这项工作中，我们应用了递归 Wasserstein 生成对抗网络（RWGAN）从体素 fMRI 数据中学习大脑表示。生成对抗网络（GAN）广泛应用于自然图像生成，可以捕获输入数据的分布，从而从 fMRI 中提取广义特征，从而缓解过拟合问题。RWGAN 的递归层旨在更好地对 fMRI 时间序列的局部时间特征进行建模。RWGAN 的鉴别器作为一个深度特征提取器。通过 LASSO 回归，RWGAN 模型可以将 fMRI 数据分解为时间特征和空间特征（FBN）。此外，RWGAN 的生成器可以为 fMRI 扩充生成高质量的新数据。
在 HCP 数据集的七个任务上的实验结果表明，与 HCP 任务设计和基于广义线性模型（GLM）的网络相比，RWGAN 可以学习到有意义和可解释的时间特征和 FBN。此外，在不同的训练数据集上的结果表明，RWGAN 在小数据集上的表现优于其他深度学习模型。此外，我们使用 RWGAN 的生成器生成虚假的个体。结果表明，与从真实数据中学习到的特征相比，虚假数据也可以用于学习有意义的表示。
据我们所知，这是最早尝试应用生成式深度学习对 fMRI 数据进行建模的工作之一。所提出的 RWGAN 为从 fMRI 数据中学习大脑表示提供了一种新的方法，它可以生成高质量的虚假数据，为 fMRI 数据扩充的潜在用途提供支持。
