Center for Information and Neural Networks, National Institute of Information and Communications Technology, Suita, Japan; Lyon Neuroscience Research Center (CRNL), INSERM U1028 - CNRS UMR5292, University of Lyon, Bron, France.
Center for Information and Neural Networks, National Institute of Information and Communications Technology, Suita, Japan; Graduate School of Frontier Biosciences, Osaka University, Suita, Japan; Graduate School of Medicine, Osaka University, Suita, Japan.
Neuroimage. 2023 Apr 15;270:119980. doi: 10.1016/j.neuroimage.2023.119980. Epub 2023 Feb 26.
Mathematical operations have long been regarded as a sparse, symbolic process in neuroimaging studies. In contrast, advances in artificial neural networks (ANN) have enabled extracting distributed representations of mathematical operations. Recent neuroimaging studies have compared distributed representations of the visual, auditory and language domains in ANNs and biological neural networks (BNNs). However, such a relationship has not yet been examined in mathematics. Here we hypothesise that ANN-based distributed representations can explain brain activity patterns of symbolic mathematical operations. We used the fMRI data of a series of mathematical problems with nine different combinations of operators to construct voxel-wise encoding/decoding models using both sparse operator and latent ANN features. Representational similarity analysis demonstrated shared representations between ANN and BNN, an effect particularly evident in the intraparietal sulcus. Feature-brain similarity (FBS) analysis served to reconstruct a sparse representation of mathematical operations based on distributed ANN features in each cortical voxel. Such reconstruction was more efficient when using features from deeper ANN layers. Moreover, latent ANN features allowed the decoding of novel operators not used during model training from brain activity. The current study provides novel insights into the neural code underlying mathematical thought.
在神经影像学研究中,数学运算长期以来一直被视为一种稀疏的、符号化的过程。相比之下,人工神经网络(ANN)的进步使得提取数学运算的分布式表示成为可能。最近的神经影像学研究比较了 ANN 和生物神经网络(BNN)中视觉、听觉和语言领域的分布式表示。然而,这种关系在数学领域还没有被研究过。在这里,我们假设基于 ANN 的分布式表示可以解释符号数学运算的大脑活动模式。我们使用了一系列具有九种不同运算符组合的数学问题的 fMRI 数据,使用稀疏运算符和潜在的 ANN 特征来构建体素级别的编码/解码模型。代表性相似性分析表明,ANN 和 BNN 之间存在共同的表示,这种效应在顶内沟尤为明显。特征-大脑相似性(FBS)分析用于根据每个皮质体素中的分布式 ANN 特征来重建数学运算的稀疏表示。当使用来自更深层 ANN 层的特征时,这种重建更加有效。此外,潜在的 ANN 特征允许从大脑活动中解码在模型训练中未使用的新运算符。本研究为数学思维的神经编码提供了新的见解。