Cheng Zhenbo, Deng Zhidong, Hu Xiaolin, Zhang Bo, Yang Tianming
State Key Laboratory of Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China; Department of Computer Science and Technology, Zhejiang University of Technology, Hangzhou, China; and.
State Key Laboratory of Intelligent Technology and Systems, Tsinghua National Laboratory for Information Science and Technology, Department of Computer Science and Technology, Tsinghua University, Beijing, China;
J Neurophysiol. 2015 Dec;114(6):3296-305. doi: 10.1152/jn.00378.2015. Epub 2015 Oct 7.
The brain often has to make decisions based on information stored in working memory, but the neural circuitry underlying working memory is not fully understood. Many theoretical efforts have been focused on modeling the persistent delay period activity in the prefrontal areas that is believed to represent working memory. Recent experiments reveal that the delay period activity in the prefrontal cortex is neither static nor homogeneous as previously assumed. Models based on reservoir networks have been proposed to model such a dynamical activity pattern. The connections between neurons within a reservoir are random and do not require explicit tuning. Information storage does not depend on the stable states of the network. However, it is not clear how the encoded information can be retrieved for decision making with a biologically realistic algorithm. We therefore built a reservoir-based neural network to model the neuronal responses of the prefrontal cortex in a somatosensory delayed discrimination task. We first illustrate that the neurons in the reservoir exhibit a heterogeneous and dynamical delay period activity observed in previous experiments. Then we show that a cluster population circuit decodes the information from the reservoir with a winner-take-all mechanism and contributes to the decision making. Finally, we show that the model achieves a good performance rapidly by shaping only the readout with reinforcement learning. Our model reproduces important features of previous behavior and neurophysiology data. We illustrate for the first time how task-specific information stored in a reservoir network can be retrieved with a biologically plausible reinforcement learning training scheme.
大脑常常需要依据工作记忆中存储的信息做出决策，但工作记忆背后的神经回路尚未被完全理解。许多理论研究致力于对前额叶区域中持续的延迟期活动进行建模，这种活动被认为代表着工作记忆。最近的实验表明，前额叶皮层中的延迟期活动既不像之前所假设的那样静止不变，也并非均匀一致。基于储层网络的模型已被提出用于对这种动态活动模式进行建模。储层内神经元之间的连接是随机的，不需要进行明确的调整。信息存储并不依赖于网络的稳定状态。然而，目前尚不清楚如何通过一种符合生物学现实的算法来检索编码信息以用于决策。因此，我们构建了一个基于储层的神经网络，以对体感延迟辨别任务中前额叶皮层的神经元反应进行建模。我们首先表明，储层中的神经元呈现出先前实验中观察到的异质性和动态延迟期活动。然后我们展示了一个聚类群体回路通过胜者全得机制对来自储层的信息进行解码，并有助于决策。最后，我们表明该模型仅通过强化学习对读出进行塑造就能快速实现良好的性能。我们的模型重现了先前行为和神经生理学数据的重要特征。我们首次说明了如何通过一种具有生物学合理性的强化学习训练方案来检索储层网络中存储的特定任务信息。
