Department of Neuroinformatics, ATR Computational Neuroscience Laboratories, Kyoto, Japan.
Graduate School of Informatics, Kyoto University, Kyoto, Japan.
Commun Biol. 2022 Jan 11;5(1):34. doi: 10.1038/s42003-021-02975-5.
Stimulus images can be reconstructed from visual cortical activity. However, our perception of stimuli is shaped by both stimulus-induced and top-down processes, and it is unclear whether and how reconstructions reflect top-down aspects of perception. Here, we investigate the effect of attention on reconstructions using fMRI activity measured while subjects attend to one of two superimposed images. A state-of-the-art method is used for image reconstruction, in which brain activity is translated (decoded) to deep neural network (DNN) features of hierarchical layers then to an image. Reconstructions resemble the attended rather than unattended images. They can be modeled by superimposed images with biased contrasts, comparable to the appearance during attention. Attentional modulations are found in a broad range of hierarchical visual representations and mirror the brain-DNN correspondence. Our results demonstrate that top-down attention counters stimulus-induced responses, modulating neural representations to render reconstructions in accordance with subjective appearance.
刺激图像可以从视觉皮层活动中重建。然而，我们对刺激的感知既受到刺激诱导的影响，也受到自上而下的过程的影响，目前尚不清楚重建是否以及如何反映感知的自上而下方面。在这里，我们使用 fMRI 活动来研究注意对重建的影响，当受试者关注两个叠加图像中的一个时，会测量到 fMRI 活动。使用最先进的方法进行图像重建，其中大脑活动被转化（解码）为分层深度神经网络（DNN）特征，然后转化为图像。重建类似于关注而不是不关注的图像。它们可以通过具有偏置对比度的叠加图像进行建模，类似于注意力期间的外观。在广泛的分层视觉表示中发现了注意力调制，并反映了大脑-DNN 的对应关系。我们的结果表明，自上而下的注意力可以抵消刺激诱导的反应，调节神经表示，使重建符合主观外观。
