RIKEN Center for Brain Science, Wako-shi, Japan.
University of Tokyo, Graduate School of Information Science and Technology, Department of Mathematical Informatics, Tokyo, Japan.
PLoS Comput Biol. 2022 Mar 14;18(3):e1009928. doi: 10.1371/journal.pcbi.1009928. eCollection 2022 Mar.
Our ability to perceive a stable visual world in the presence of continuous movements of the body, head, and eyes has puzzled researchers in the neuroscience field for a long time. We reformulated this problem in the context of hierarchical convolutional neural networks (CNNs)-whose architectures have been inspired by the hierarchical signal processing of the mammalian visual system-and examined perceptual stability as an optimization process that identifies image-defining features for accurate image classification in the presence of movements. Movement signals, multiplexed with visual inputs along overlapping convolutional layers, aided classification invariance of shifted images by making the classification faster to learn and more robust relative to input noise. Classification invariance was reflected in activity manifolds associated with image categories emerging in late CNN layers and with network units acquiring movement-associated activity modulations as observed experimentally during saccadic eye movements. Our findings provide a computational framework that unifies a multitude of biological observations on perceptual stability under optimality principles for image classification in artificial neural networks.
我们能够在身体、头部和眼睛持续运动的情况下感知到稳定的视觉世界，这一现象长期以来一直困扰着神经科学领域的研究人员。我们在分层卷积神经网络（CNN）的背景下重新表述了这个问题——其架构受到了哺乳动物视觉系统分层信号处理的启发——并将感知稳定性视为一种优化过程，该过程确定了用于在运动存在的情况下进行准确图像分类的图像定义特征。运动信号与视觉输入沿着重叠的卷积层多路复用，通过使分类更快地学习并相对于输入噪声更稳健，从而帮助实现了移位图像的分类不变性。分类不变性反映在与晚期 CNN 层中出现的图像类别相关联的活动流形中，以及与实验中在扫视眼动期间观察到的与运动相关的活动调制相关联的网络单元中。我们的发现提供了一个计算框架，该框架统一了大量关于在人工神经网络中进行图像分类的最优性原则下的感知稳定性的生物学观察。
