Yamada Kentaro, Miyawaki Yoichi, Kamitani Yukiyasu
Fundamental Technology Research Center, Honda R&D Co., Ltd., Saitama 351-0188, Japan; ATR Computational Neuroscience Laboratories, Kyoto 619-0288, Japan.
National Institute of Information and Communications Technology, Kyoto 619-0288, Japan; ATR Computational Neuroscience Laboratories, Kyoto 619-0288, Japan; The University of Electro-Communications, Tokyo 182-8585, Japan.
Neuroimage. 2015 Jun;113:289-97. doi: 10.1016/j.neuroimage.2015.03.059. Epub 2015 Apr 2.
Brain activity patterns differ from person to person, even for an identical stimulus. In functional brain mapping studies, it is important to align brain activity patterns between subjects for group statistical analyses. While anatomical templates are widely used for inter-subject alignment in functional magnetic resonance imaging (fMRI) studies, they are not sufficient to identify the mapping between voxel-level functional responses representing specific mental contents. Recent work has suggested that statistical learning methods could be used to transform individual brain activity patterns into a common space while preserving representational contents. Here, we propose a flexible method for functional alignment, "neural code converter," which converts one subject's brain activity pattern into another's representing the same content. The neural code converter was designed to learn statistical relationships between fMRI activity patterns of paired subjects obtained while they saw an identical series of stimuli. It predicts the signal intensity of individual voxels of one subject from a pattern of multiple voxels of the other subject. To test this method, we used fMRI activity patterns measured while subjects observed visual images consisting of random and structured patches. We show that fMRI activity patterns for visual images not used for training the converter could be predicted from those of another subject where brain activity was recorded for the same stimuli. This confirms that visual images can be accurately reconstructed from the predicted activity patterns alone. Furthermore, we show that a classifier trained only on predicted fMRI activity patterns could accurately classify measured fMRI activity patterns. These results demonstrate that the neural code converter can translate neural codes between subjects while preserving contents related to visual images. While this method is useful for functional alignment and decoding, it may also provide a basis for brain-to-brain communication using the converted pattern for designing brain stimulation.
即使对于相同的刺激，大脑活动模式也因人而异。在功能性脑图谱研究中，为了进行组统计分析，使受试者之间的大脑活动模式对齐非常重要。虽然解剖模板在功能磁共振成像（fMRI）研究中被广泛用于受试者间对齐，但它们不足以识别代表特定心理内容的体素级功能反应之间的映射关系。最近的研究表明，统计学习方法可用于将个体大脑活动模式转换到一个公共空间，同时保留其表征内容。在此，我们提出一种灵活的功能对齐方法——“神经编码转换器”，它能将一个受试者的大脑活动模式转换为另一个受试者代表相同内容的模式。神经编码转换器旨在学习配对受试者在观看一系列相同刺激时获得的fMRI活动模式之间的统计关系。它根据另一个受试者多个体素的模式预测一个受试者个体体素的信号强度。为了测试该方法，我们使用了受试者观察由随机和结构化斑块组成的视觉图像时测量的fMRI活动模式。我们表明，对于未用于训练转换器的视觉图像的fMRI活动模式，可以从另一个对相同刺激进行大脑活动记录的受试者的模式中预测出来。这证实了仅从预测的活动模式就能准确重建视觉图像。此外，我们表明仅在预测的fMRI活动模式上训练的分类器能够准确地对测量的fMRI活动模式进行分类。这些结果表明，神经编码转换器可以在受试者之间转换神经编码，同时保留与视觉图像相关的内容。虽然这种方法对于功能对齐和解码很有用，但它也可能为使用转换后的模式进行脑刺激设计的脑对脑通信提供基础。
