Suppr超能文献

混合策略游戏中的顶内沟外侧皮质与强化学习

Lateral intraparietal cortex and reinforcement learning during a mixed-strategy game.

作者信息

Seo Hyojung, Barraclough Dominic J, Lee Daeyeol

机构信息

Department of Neurobiology, Yale University School of Medicine, New Haven, Connecticut 06510, USA.

出版信息

J Neurosci. 2009 Jun 3;29(22):7278-89. doi: 10.1523/JNEUROSCI.1479-09.2009.

Abstract

Activity of the neurons in the lateral intraparietal cortex (LIP) displays a mixture of sensory, motor, and memory signals. Moreover, they often encode signals reflecting the accumulation of sensory evidence that certain eye movements might lead to a desirable outcome. However, when the environment changes dynamically, animals are also required to combine the information about its previously chosen actions and their outcomes appropriately to update continually the desirabilities of alternative actions. Here, we investigated whether LIP neurons encoded signals necessary to update an animal's decision-making strategies adaptively during a computer-simulated matching-pennies game. Using a reinforcement learning algorithm, we estimated the value functions that best predicted the animal's choices on a trial-by-trial basis. We found that, immediately before the animal revealed its choice, approximately 18% of LIP neurons changed their activity according to the difference in the value functions for the two targets. In addition, a somewhat higher fraction of LIP neurons displayed signals related to the sum of the value functions, which might correspond to the state value function or an average rate of reward used as a reference point. Similar to the neurons in the prefrontal cortex, many LIP neurons also encoded the signals related to the animal's previous choices. Thus, the posterior parietal cortex might be a part of the network that provides the substrate for forming appropriate associations between actions and outcomes.

摘要

顶内沟外侧皮质(LIP)中的神经元活动呈现出感觉、运动和记忆信号的混合。此外,它们常常编码反映感觉证据积累的信号,即某些眼动可能会带来理想的结果。然而,当环境动态变化时,动物还需要适当地整合有关其先前选择的行动及其结果的信息,以便不断更新替代行动的可取性。在此,我们研究了在计算机模拟的猜硬币游戏中,LIP神经元是否编码了动物自适应更新决策策略所需的信号。我们使用强化学习算法,逐次估计最能预测动物选择的价值函数。我们发现,就在动物做出选择之前,约18%的LIP神经元根据两个目标的价值函数差异改变了它们的活动。此外,略高比例的LIP神经元显示出与价值函数总和相关的信号,这可能对应于状态价值函数或用作参考点的平均奖励率。与前额叶皮质中的神经元类似,许多LIP神经元也编码了与动物先前选择相关的信号。因此,顶叶后皮质可能是为行动和结果之间形成适当关联提供基础的神经网络的一部分。

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验