Khaligh-Razavi Seyed-Mahdi, Kriegeskorte Nikolaus
Medical Research Council, Cognition and Brain Sciences Unit, Cambridge, United Kingdom.
PLoS Comput Biol. 2014 Nov 6;10(11):e1003915. doi: 10.1371/journal.pcbi.1003915. eCollection 2014 Nov.
Inferior temporal (IT) cortex in human and nonhuman primates serves visual object recognition. Computational object-vision models, although continually improving, do not yet reach human performance. It is unclear to what extent the internal representations of computational models can explain the IT representation. Here we investigate a wide range of computational model representations (37 in total), testing their categorization performance and their ability to account for the IT representational geometry. The models include well-known neuroscientific object-recognition models (e.g. HMAX, VisNet) along with several models from computer vision (e.g. SIFT, GIST, self-similarity features, and a deep convolutional neural network). We compared the representational dissimilarity matrices (RDMs) of the model representations with the RDMs obtained from human IT (measured with fMRI) and monkey IT (measured with cell recording) for the same set of stimuli (not used in training the models). Better performing models were more similar to IT in that they showed greater clustering of representational patterns by category. In addition, better performing models also more strongly resembled IT in terms of their within-category representational dissimilarities. Representational geometries were significantly correlated between IT and many of the models. However, the categorical clustering observed in IT was largely unexplained by the unsupervised models. The deep convolutional network, which was trained by supervision with over a million category-labeled images, reached the highest categorization performance and also best explained IT, although it did not fully explain the IT data. Combining the features of this model with appropriate weights and adding linear combinations that maximize the margin between animate and inanimate objects and between faces and other objects yielded a representation that fully explained our IT data. Overall, our results suggest that explaining IT requires computational features trained through supervised learning to emphasize the behaviorally important categorical divisions prominently reflected in IT.
人类和非人类灵长类动物的颞下回（IT）皮质负责视觉物体识别。尽管计算物体视觉模型在不断改进，但仍未达到人类的表现水平。目前尚不清楚计算模型的内部表征在多大程度上能够解释IT表征。在此，我们研究了广泛的计算模型表征（总共37种），测试它们的分类性能以及解释IT表征几何结构的能力。这些模型包括著名的神经科学物体识别模型（如HMAX、VisNet）以及计算机视觉领域的几种模型（如尺度不变特征变换（SIFT）、全局特征描述符（GIST）、自相似特征和深度卷积神经网络）。我们将模型表征的表征差异矩阵（RDM）与从人类IT（通过功能磁共振成像（fMRI）测量）和猴子IT（通过细胞记录测量）获得的RDM进行了比较，这些RDM是针对同一组刺激（未用于模型训练）。表现较好的模型与IT更为相似，因为它们按类别显示出更大的表征模式聚类。此外，表现较好的模型在类别内表征差异方面也与IT更为相似。IT与许多模型之间的表征几何结构显著相关。然而，IT中观察到的类别聚类在很大程度上无法由无监督模型解释。通过对超过一百万个带有类别标签的图像进行监督训练的深度卷积神经网络达到了最高的分类性能，并且对IT的解释也最好，尽管它没有完全解释IT数据。将该模型的特征与适当的权重相结合，并添加能够最大化有生命与无生命物体之间以及面部与其他物体之间边界的线性组合，得到了一种能够完全解释我们IT数据的表征。总体而言，我们的结果表明，解释IT需要通过监督学习训练的计算特征，以突出在IT中显著体现的行为上重要的类别划分。
