Department of Psychology, Yale University, New Haven, Connecticut 06520
National Institute of Mental Health, Bethesda, Maryland 20892-9663.
J Neurosci. 2021 May 12;41(19):4234-4252. doi: 10.1523/JNEUROSCI.1993-20.2021. Epub 2021 Mar 31.
A visual object is characterized by multiple visual features, including its identity, position and size. Despite the usefulness of identity and nonidentity features in vision and their joint coding throughout the primate ventral visual processing pathway, they have so far been studied relatively independently. Here in both female and male human participants, the coding of identity and nonidentity features was examined together across the human ventral visual pathway. The nonidentity features tested included two Euclidean features (position and size) and two non-Euclidean features (image statistics and spatial frequency (SF) content of an image). Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with identity outweighing the non-Euclidean but not the Euclidean features at higher levels of visual processing. In 14 convolutional neural networks (CNNs) pretrained for object categorization with varying architecture, depth, and with/without recurrent processing, nonidentity feature representation showed an initial large increase from early to mid-stage of processing, followed by a decrease at later stages of processing, different from brain responses. Additionally, from lower to higher levels of visual processing, position became more underrepresented and image statistics and SF became more overrepresented compared with identity in CNNs than in the human brain. Similar results were obtained in a CNN trained with stylized images that emphasized shape representations. Overall, by measuring the coding strength of object identity and nonidentity features together, our approach provides a new tool for characterizing feature coding in the human brain and the correspondence between the brain and CNNs. This study examined the coding strength of object identity and four types of nonidentity features along the human ventral visual processing pathway and compared brain responses with those of 14 convolutional neural networks (CNNs) pretrained to perform object categorization. Overall, identity representation increased and nonidentity feature representation decreased along the ventral visual pathway, with some notable differences among the different nonidentity features. CNNs differed from the brain in a number of aspects in their representations of identity and nonidentity features over the course of visual processing. Our approach provides a new tool for characterizing feature coding in the human brain and the correspondence between the brain and CNNs.
视觉物体的特征包括其身份、位置和大小等多种视觉特征。尽管身份和非身份特征在视觉中具有重要作用，并且在灵长类动物腹侧视觉处理途径中联合编码，但迄今为止，它们的研究相对独立。本研究在女性和男性人类参与者中，共同研究了身份和非身份特征在人类腹侧视觉途径中的编码。测试的非身份特征包括两个欧几里得特征（位置和大小）和两个非欧几里得特征（图像统计和图像空间频率（SF）内容）。总体而言，身份表示随着腹侧视觉通路的增加而增加，非身份特征表示随着视觉处理水平的提高而减少，身份特征比非欧几里得特征但不比欧几里得特征更重要。在 14 个用于对象分类的卷积神经网络（CNN）中，通过不同的架构、深度以及是否具有递归处理来预训练，非身份特征表示在处理的早期到中期呈现出初始的大幅增加，然后在后期阶段下降，与大脑反应不同。此外，与大脑反应相比，从较低到较高的视觉处理水平，位置表示变得更加不足，图像统计和 SF 表示变得更加过度代表，在 CNN 中比在大脑中更明显。在使用强调形状表示的风格化图像训练的 CNN 中，也得到了类似的结果。总体而言，通过一起测量对象身份和非身份特征的编码强度，我们的方法为描述大脑中的特征编码以及大脑和 CNN 之间的对应关系提供了新的工具。
