Xu Aiwen, Hou Yuchen, Niell Cristopher M, Beyeler Michael
Department of Computer Science University of California, Santa Barbara Santa Barbara, CA 93117.
Department of Biology, Institute of Neuroscience University of Oregon Eugene, OR 97403.
Adv Neural Inf Process Syst. 2023 Dec;36:15341-15357.
Despite their immense success as a model of macaque visual cortex, deep convolutional neural networks (CNNs) have struggled to predict activity in visual cortex of the mouse, which is thought to be strongly dependent on the animal's behavioral state. Furthermore, most computational models focus on predicting neural responses to static images presented under head fixation, which are dramatically different from the dynamic, continuous visual stimuli that arise during movement in the real world. Consequently, it is still unknown how natural visual input and different behavioral variables may integrate over time to generate responses in primary visual cortex (V1). To address this, we introduce a multimodal recurrent neural network that integrates gaze-contingent visual input with behavioral and temporal dynamics to explain V1 activity in freely moving mice. We show that the model achieves state-of-the-art predictions of V1 activity during free exploration and demonstrate the importance of each component in an extensive ablation study. Analyzing our model using maximally activating stimuli and saliency maps, we reveal new insights into cortical function, including the prevalence of mixed selectivity for behavioral variables in mouse V1. In summary, our model offers a comprehensive deep-learning framework for exploring the computational principles underlying V1 neurons in freely-moving animals engaged in natural behavior.
尽管深度卷积神经网络（CNNs）作为猕猴视觉皮层的模型取得了巨大成功，但它们在预测小鼠视觉皮层的活动方面却遇到了困难，因为小鼠视觉皮层的活动被认为强烈依赖于动物的行为状态。此外，大多数计算模型专注于预测在头部固定状态下呈现的静态图像的神经反应，而这与现实世界中动物移动时出现的动态、连续视觉刺激有很大不同。因此，自然视觉输入和不同行为变量如何随时间整合以在初级视觉皮层（V1）中产生反应仍然未知。为了解决这个问题，我们引入了一种多模态循环神经网络，该网络将与注视相关的视觉输入与行为和时间动态相结合，以解释自由移动小鼠的V1活动。我们表明，该模型在自由探索期间实现了对V1活动的最先进预测，并在广泛的消融研究中证明了每个组件的重要性。通过使用最大激活刺激和显著性图分析我们的模型，我们揭示了对皮层功能的新见解，包括小鼠V1中行为变量混合选择性的普遍性。总之，我们的模型提供了一个全面的深度学习框架，用于探索参与自然行为的自由移动动物中V1神经元的计算原理。
