Faculty of Behavior and Movement Sciences, Vrije Universiteit Amsterdam, 1081 BT Amsterdam, Noord Holland, The Netherlands
Machine Learning Group, Centrum voor Wiskunde & Informatica, 1098 XG Amsterdam, Noord Holland, The Netherlands; Swammerdam Institute of Life Sciences, University of Amsterdam, 1098 XH Amsterdam, Noord Holland, The Netherlands; and Department of Computer Science, Rijksuniversiteit Groningen, 9747 AG Groningen, The Netherlands
Neural Comput. 2021 Jan;33(1):1-40. doi: 10.1162/neco_a_01339. Epub 2020 Oct 20.
Working memory is essential: it serves to guide intelligent behavior of humans and nonhuman primates when task-relevant stimuli are no longer present to the senses. Moreover, complex tasks often require that multiple working memory representations can be flexibly and independently maintained, prioritized, and updated according to changing task demands. Thus far, neural network models of working memory have been unable to offer an integrative account of how such control mechanisms can be acquired in a biologically plausible manner. Here, we present WorkMATe, a neural network architecture that models cognitive control over working memory content and learns the appropriate control operations needed to solve complex working memory tasks. Key components of the model include a gated memory circuit that is controlled by internal actions, encoding sensory information through untrained connections, and a neural circuit that matches sensory inputs to memory content. The network is trained by means of a biologically plausible reinforcement learning rule that relies on attentional feedback and reward prediction errors to guide synaptic updates. We demonstrate that the model successfully acquires policies to solve classical working memory tasks, such as delayed recognition and delayed pro-saccade/anti-saccade tasks. In addition, the model solves much more complex tasks, including the hierarchical 12-AX task or the ABAB ordered recognition task, both of which demand an agent to independently store and updated multiple items separately in memory. Furthermore, the control strategies that the model acquires for these tasks subsequently generalize to new task contexts with novel stimuli, thus bringing symbolic production rule qualities to a neural network architecture. As such, WorkMATe provides a new solution for the neural implementation of flexible memory control.
当与任务相关的刺激不再出现在感官中时，它有助于指导人类和非人类灵长类动物的智能行为。此外，复杂的任务通常需要能够灵活且独立地保持、优先处理和根据不断变化的任务需求更新多个工作记忆表示。到目前为止，工作记忆的神经网络模型还无法提供一个综合的解释，说明这些控制机制如何以生物上合理的方式获得。在这里，我们提出了 WorkMATe，这是一种神经网络架构，可对工作记忆内容进行认知控制，并学习解决复杂工作记忆任务所需的适当控制操作。该模型的关键组件包括一个门控记忆电路，该电路由内部动作控制，通过未经训练的连接对感官信息进行编码，以及一个将感官输入与记忆内容匹配的神经电路。该网络通过一种基于注意力反馈和奖励预测误差的生物合理的强化学习规则进行训练，以指导突触更新。我们证明该模型成功地获取了解决经典工作记忆任务的策略，例如延迟识别和延迟前向/反向眼跳任务。此外，该模型还解决了更复杂的任务，包括分层 12-AX 任务或 ABAB 有序识别任务，这些任务都需要一个代理在记忆中分别独立地存储和更新多个项目。此外，该模型为这些任务获取的控制策略随后推广到具有新刺激的新任务环境中，从而使符号产生规则的特性应用于神经网络架构。因此，WorkMATe 为灵活的记忆控制的神经实现提供了一种新的解决方案。
