Kazemivash Behnam, Suresh Pranav Nadigapu, Ye Dong Hye, Iraji Armin, Liu Jingyu, Plis Sergey, Kochunov Peter, Calhoun Vince
bioRxiv. 2024 Nov 28:2024.11.28.625914. doi: 10.1101/2024.11.28.625914.
Modeling dynamic neuronal activity within brain networks enables the precise tracking of rapid temporal fluctuations across different brain regions. However, current approaches in computational neuroscience fall short of capturing and representing the spatiotemporal dynamics within each brain network. We developed a novel weakly supervised spatiotemporal dense prediction model capable of generating personalized 4D dynamic brain networks from fMRI data, providing a more granular representation of brain activity over time.
We developed a model that leverages the vision transformer (ViT) as its backbone, jointly encoding spatial and temporal information from fMRI inputs using two different configurations: space-time and sequential encoders. The model generates 4D brain network maps that evolve over time, capturing dynamic changes in both spatial and temporal dimensions. In the absence of ground-truth data, we used spatially constrained windowed independent component analysis (ICA) components derived from fMRI data as weak supervision to guide the training process. The model was evaluated using large-scale resting-state fMRI datasets, and statistical analyses were conducted to assess the effectiveness of the generated dynamic maps using various metrics.
Our model effectively produced 4D brain maps that captured both inter-subject and temporal variations, offering a dynamic representation of evolving brain networks. Notably, the model demonstrated the ability to produce smooth maps from noisy priors, effectively denoising the resulting brain dynamics. Additionally, statistically significant differences were observed in the temporally averaged brain maps, as well as in the summation of absolute temporal gradient maps, between patients with schizophrenia and healthy controls. For example, within the Default Mode Network (DMN), significant differences emerged in the temporally averaged space-time configurations, particularly in the thalamus, where healthy controls exhibited higher activity levels compared to subjects with schizophrenia. These findings highlight the model's potential for differentiating between clinical populations.
The proposed spatiotemporal dense prediction model offers an effective approach for generating dynamic brain maps by capturing significant spatiotemporal variations in brain activity. Leveraging weak supervision through ICA components enables the model to learn dynamic patterns without direct ground-truth data, making it a robust and efficient tool for brain mapping.
This work presents an important new approach for dynamic brain mapping, potentially opening up new opportunities for studying brain dynamics within specific networks. By framing the problem as a spatiotemporal dense prediction task in computer vision, we leverage the spatiotemporal ViT architecture combined with weakly supervised learning techniques to efficiently and effectively estimate these maps.
对脑网络内的动态神经元活动进行建模，能够精确追踪不同脑区快速的时间波动。然而，计算神经科学中的当前方法在捕捉和表示每个脑网络内的时空动态方面存在不足。我们开发了一种新型的弱监督时空密集预测模型，该模型能够根据功能磁共振成像（fMRI）数据生成个性化的4D动态脑网络，提供随时间变化的更精细的脑活动表示。
我们开发了一种以视觉Transformer（ViT）为骨干的模型，使用两种不同配置（时空编码器和顺序编码器）联合编码来自fMRI输入的空间和时间信息。该模型生成随时间演变的4D脑网络图，捕捉空间和时间维度上的动态变化。在没有真实数据的情况下，我们使用从fMRI数据中导出的空间约束窗口独立成分分析（ICA）成分作为弱监督来指导训练过程。使用大规模静息态fMRI数据集对该模型进行评估，并进行统计分析以使用各种指标评估生成的动态图的有效性。
我们的模型有效地生成了捕捉个体间和时间变化的4D脑图，提供了不断演变的脑网络的动态表示。值得注意的是，该模型展示了从有噪声的先验数据生成平滑图的能力，有效地对所得脑动态进行去噪。此外，在精神分裂症患者和健康对照之间，在时间平均脑图以及绝对时间梯度图的总和中观察到统计学上的显著差异。例如，在默认模式网络（DMN）内，时间平均时空配置出现了显著差异，特别是在丘脑，与精神分裂症患者相比，健康对照表现出更高的活动水平。这些发现突出了该模型区分临床群体的潜力。
所提出的时空密集预测模型通过捕捉脑活动中显著的时空变化，为生成动态脑图提供了一种有效方法。通过ICA成分利用弱监督使模型能够在没有直接真实数据的情况下学习动态模式，使其成为脑图谱绘制的强大而高效的工具。
这项工作提出了一种重要的动态脑图谱绘制新方法，可能为研究特定网络内的脑动态开辟新机会。通过将该问题构建为计算机视觉中的时空密集预测任务，我们利用时空ViT架构结合弱监督学习技术来高效且有效地估计这些图。
