Ahn Seoyoung, Adeli Hossein, Zelinsky Gregory J
Department of Molecular and Cell Biology, University of California, Berkeley, California, United States of America.
Zuckerman Mind Brain Behavior Institute, Columbia University, New York City, New York, United States of America.
PLoS Comput Biol. 2024 Jun 13;20(6):e1012159. doi: 10.1371/journal.pcbi.1012159. eCollection 2024 Jun.
Humans are extremely robust in our ability to perceive and recognize objects-we see faces in tea stains and can recognize friends on dark streets. Yet, neurocomputational models of primate object recognition have focused on the initial feed-forward pass of processing through the ventral stream and less on the top-down feedback that likely underlies robust object perception and recognition. Aligned with the generative approach, we propose that the visual system actively facilitates recognition by reconstructing the object hypothesized to be in the image. Top-down attention then uses this reconstruction as a template to bias feedforward processing to align with the most plausible object hypothesis. Building on auto-encoder neural networks, our model makes detailed hypotheses about the appearance and location of the candidate objects in the image by reconstructing a complete object representation from potentially incomplete visual input due to noise and occlusion. The model then leverages the best object reconstruction, measured by reconstruction error, to direct the bottom-up process of selectively routing low-level features, a top-down biasing that captures a core function of attention. We evaluated our model using the MNIST-C (handwritten digits under corruptions) and ImageNet-C (real-world objects under corruptions) datasets. Not only did our model achieve superior performance on these challenging tasks designed to approximate real-world noise and occlusion viewing conditions, but also better accounted for human behavioral reaction times and error patterns than a standard feedforward Convolutional Neural Network. Our model suggests that a complete understanding of object perception and recognition requires integrating top-down and attention feedback, which we propose is an object reconstruction.
人类在感知和识别物体方面具有极强的能力——我们能在茶渍中看到人脸,能在黑暗的街道上认出朋友。然而,灵长类动物物体识别的神经计算模型主要关注通过腹侧流进行的初始前馈处理过程,而较少关注可能是稳健物体感知和识别基础的自上而下的反馈。与生成方法一致,我们提出视觉系统通过重建假设存在于图像中的物体来积极促进识别。然后,自上而下的注意力将此重建作为模板,使前馈处理产生偏差,以与最合理的物体假设对齐。基于自动编码器神经网络,我们的模型通过从因噪声和遮挡而可能不完整的视觉输入中重建完整的物体表示,对图像中候选物体的外观和位置做出详细假设。然后,该模型利用以重建误差衡量的最佳物体重建结果,来指导选择性路由低级特征的自下而上过程,这种自上而下的偏差捕捉了注意力的核心功能。我们使用MNIST-C(受干扰的手写数字)和ImageNet-C(受干扰的真实世界物体)数据集对我们的模型进行了评估。我们的模型不仅在这些旨在近似真实世界噪声和遮挡观看条件的具有挑战性的任务上取得了卓越的性能,而且与标准的前馈卷积神经网络相比,能更好地解释人类行为反应时间和错误模式。我们的模型表明,对物体感知和识别的全面理解需要整合自上而下和注意力反馈,我们认为这是一种物体重建。