Department of Psychology, National Taiwan University, 1 Roosevelt Road Sec. 4, Taipei, Taiwan.
Genes Brain Behav. 2012 Mar;11(2):157-69. doi: 10.1111/j.1601-183X.2011.00759.x.
In contemporary reinforcement learning models, reward prediction error (RPE), the difference between the expected and actual reward, is thought to guide action value learning through the firing activity of dopaminergic neurons. Given the importance of dopamine in reward learning and the involvement of Akt1 in dopamine-dependent behaviors, the aim of this study was to investigate whether Akt1 deficiency modulates reward learning and the magnitude of RPE using Akt1 mutant mice as a model. In comparison to wild-type littermate controls, the expression of Akt1 proteins in mouse brains occurred in a gene-dosage-dependent manner and Akt1 heterozygous (HET) mice exhibited impaired striatal Akt1 activity under methamphetamine challenge. No genotypic difference was found in the basal levels of dopamine and its metabolites. In a series of reward-related learning tasks, HET mice displayed a relatively efficient method of updating reward information from the environment during the acquisition phase of the two natural reward tasks and in the reverse section of the dynamic foraging T-maze but not in methamphetamine-induced or aversive-related reward learning. The implementation of a standard reinforcement learning model and the Bayesian hierarchical parameter estimation show that HET mice have higher RPE magnitudes and that their action values are updated more rapidly among all three test sections in T-maze. These results indicate that Akt1 deficiency modulates natural reward learning and RPE. This study showed a promising avenue for investigating RPE in mutant mice and provided evidence for the potential link from genetic deficiency, to neurobiological abnormalities, to impairment in higher-order cognitive functioning.
在当代强化学习模型中,奖励预测误差(RPE),即期望奖励与实际奖励之间的差异,被认为通过多巴胺能神经元的放电活动来指导动作价值学习。鉴于多巴胺在奖励学习中的重要性以及 Akt1 在多巴胺依赖行为中的参与,本研究旨在探讨 Akt1 缺陷是否通过 Akt1 突变小鼠模型调节奖励学习和 RPE 的幅度。与野生型同窝对照相比,Akt1 蛋白在小鼠大脑中的表达呈基因剂量依赖性,Akt1 杂合(HET)小鼠在甲基苯丙胺挑战下表现出纹状体 Akt1 活性受损。多巴胺及其代谢物的基础水平在基因型之间没有差异。在一系列与奖励相关的学习任务中,HET 小鼠在两个自然奖励任务的获取阶段以及动态觅食 T 迷宫的反转部分表现出相对有效的从环境中更新奖励信息的方法,但在甲基苯丙胺诱导的或厌恶相关的奖励学习中没有表现出这种方法。标准强化学习模型的实施和贝叶斯分层参数估计表明,HET 小鼠在 T 迷宫的所有三个测试部分中具有更高的 RPE 幅度,并且它们的动作值更新速度更快。这些结果表明 Akt1 缺陷调节自然奖励学习和 RPE。本研究为在突变小鼠中研究 RPE 提供了一个有前途的途径,并为从遗传缺陷到神经生物学异常再到高级认知功能障碍的潜在联系提供了证据。