Department of Psychology, Yale University, New Haven, CT, United States of America.
PLoS One. 2022 Jun 28;17(6):e0270667. doi: 10.1371/journal.pone.0270667. eCollection 2022.
Scene perception involves extracting the identities of the objects comprising a scene in conjunction with their configuration (the spatial layout of the objects in the scene). How object identity and configuration information is weighted during scene processing and how this weighting evolves over the course of scene processing however, is not fully understood. Recent developments in convolutional neural networks (CNNs) have demonstrated their aptitude at scene processing tasks and identified correlations between processing in CNNs and in the human brain. Here we examined four CNN architectures (Alexnet, Resnet18, Resnet50, Densenet161) and their sensitivity to changes in object and configuration information over the course of scene processing. Despite differences among the four CNN architectures, across all CNNs, we observed a common pattern in the CNN's response to object identity and configuration changes. Each CNN demonstrated greater sensitivity to configuration changes in early stages of processing and stronger sensitivity to object identity changes in later stages. This pattern persists regardless of the spatial structure present in the image background, the accuracy of the CNN in classifying the scene, and even the task used to train the CNN. Importantly, CNNs' sensitivity to a configuration change is not the same as their sensitivity to any type of position change, such as that induced by a uniform translation of the objects without a configuration change. These results provide one of the first documentations of how object identity and configuration information are weighted in CNNs during scene processing.
场景感知涉及提取构成场景的物体的身份及其配置（场景中物体的空间布局）。然而，在场景处理过程中，对象身份和配置信息是如何加权的，以及这种加权是如何随场景处理过程的发展而演变的，目前还不完全清楚。卷积神经网络（CNN）的最新发展表明了它们在场景处理任务中的能力，并确定了 CNN 与人类大脑之间的处理相关性。在这里，我们研究了四个 CNN 架构（Alexnet、Resnet18、Resnet50、Densenet161）及其对场景处理过程中对象和配置信息变化的敏感性。尽管这四个 CNN 架构存在差异，但在所有 CNN 中，我们观察到了 CNN 对对象身份和配置变化的响应存在一种共同模式。每个 CNN 在处理的早期阶段对配置变化的敏感性更强，而在后期阶段对对象身份变化的敏感性更强。这种模式与图像背景中的空间结构、CNN 对场景分类的准确性甚至用于训练 CNN 的任务无关。重要的是，CNN 对配置变化的敏感性与它们对任何类型的位置变化的敏感性不同，例如在不改变配置的情况下对物体进行均匀平移所引起的位置变化。这些结果提供了关于对象身份和配置信息在场景处理过程中在 CNN 中加权的首批文档之一。
