Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany.
International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany.
Elife. 2023 Mar 14;12:e82301. doi: 10.7554/eLife.82301.
Replay of neuronal sequences in the hippocampus during resting states and sleep play an important role in learning and memory consolidation. Consistent with these functions, replay sequences have been shown to obey current spatial constraints. Nevertheless, replay does not necessarily reflect previous behavior and can construct never-experienced sequences. Here, we propose a stochastic replay mechanism that prioritizes experiences based on three variables: 1. Experience strength, 2. experience similarity, and 3. inhibition of return. Using this prioritized replay mechanism to train reinforcement learning agents leads to far better performance than using random replay. Its performance is close to the state-of-the-art, but computationally intensive, algorithm by Mattar & Daw (2018). Importantly, our model reproduces diverse types of replay because of the stochasticity of the replay mechanism and experience-dependent differences between the three variables. In conclusion, a unified replay mechanism generates diverse replay statistics and is efficient in driving spatial learning.
在静息状态和睡眠期间，海马体中神经元序列的回放对于学习和记忆巩固起着重要作用。与这些功能一致，回放序列被证明服从当前的空间约束。然而，回放并不一定反映以前的行为，并且可以构建从未经历过的序列。在这里，我们提出了一种随机回放机制，该机制根据三个变量对经验进行优先级排序：1.经验强度，2.经验相似性，3.返回抑制。使用这种优先级回放机制来训练强化学习代理可以获得比随机回放更好的性能。它的性能接近 Mattar 和 Daw (2018) 的最先进的、但计算密集型的算法。重要的是，由于回放机制的随机性和三个变量之间的经验依赖性差异，我们的模型再现了多种类型的回放。总之，统一的回放机制产生了多样化的回放统计数据，并有效地驱动空间学习。
