Kirubeswaran O R, Storrs Katherine R
Indian Institute of Science Education and Research Pune, India.
Department of Experimental Psychology, Justus Liebig University Giessen, Germany; Centre for Mind, Brain and Behaviour (CMBB), University of Marburg and Justus Liebig University Giessen, Germany; School of Psychology, University of Auckland, New Zealand.
Vision Res. 2023 May;206:108195. doi: 10.1016/j.visres.2023.108195. Epub 2023 Feb 17.
Why do we perceive illusory motion in some static images? Several accounts point to eye movements, response latencies to different image elements, or interactions between image patterns and motion energy detectors. Recently PredNet, a recurrent deep neural network (DNN) based on predictive coding principles, was reported to reproduce the "Rotating Snakes" illusion, suggesting a role for predictive coding. We begin by replicating this finding, then use a series of "in silico" psychophysics and electrophysiology experiments to examine whether PredNet behaves consistently with human observers and non-human primate neural data. A pretrained PredNet predicted illusory motion for all subcomponents of the Rotating Snakes pattern, consistent with human observers. However, we found no simple response delays in internal units, unlike evidence from electrophysiological data. PredNet's detection of motion in gradients seemed dependent on contrast, but depends predominantly on luminance in humans. Finally, we examined the robustness of the illusion across ten PredNets of identical architecture, retrained on the same video data. There was large variation across network instances in whether they reproduced the Rotating Snakes illusion, and what motion, if any, they predicted for simplified variants. Unlike human observers, no network predicted motion for greyscale variants of the Rotating Snakes pattern. Our results sound a cautionary note: even when a DNN successfully reproduces some idiosyncrasy of human vision, more detailed investigation can reveal inconsistencies between humans and the network, and between different instances of the same network. These inconsistencies suggest that predictive coding does not reliably give rise to human-like illusory motion.
为什么我们会在一些静态图像中感知到虚幻运动？有几种解释指向了眼球运动、对不同图像元素的反应潜伏期，或者图像模式与运动能量探测器之间的相互作用。最近，据报道基于预测编码原理的循环深度神经网络（DNN）PredNet能够重现“旋转蛇”错觉，这表明预测编码发挥了作用。我们首先复制了这一发现，然后使用一系列“计算机模拟”心理物理学和电生理学实验来检验PredNet的行为是否与人类观察者以及非人类灵长类动物的神经数据一致。一个经过预训练的PredNet对旋转蛇图案的所有子成分都预测出了虚幻运动，这与人类观察者的情况一致。然而，与电生理数据的证据不同，我们在内部单元中没有发现简单的反应延迟。PredNet对梯度中运动的检测似乎取决于对比度，但在人类中主要取决于亮度。最后，我们在十个具有相同架构的PredNet上进行了重新训练，这些网络都使用相同的视频数据，以此检验错觉的稳健性。在是否能重现旋转蛇错觉以及它们对简化变体预测出何种运动（如果有的话）方面，不同网络实例之间存在很大差异。与人类观察者不同，没有一个网络能预测出旋转蛇图案灰度变体的运动。我们的结果敲响了警钟：即使一个深度神经网络成功地重现了人类视觉的某些特质，更详细的研究也可能揭示出人类与该网络之间以及同一网络的不同实例之间的不一致。这些不一致表明，预测编码并不能可靠地产生类似人类的虚幻运动。
