Suppr超能文献

价值驱动的中脑边缘多巴胺释放的适应性变化受基于模型和无模型机制的共同调控。

Value-Driven Adaptations of Mesolimbic Dopamine Release Are Governed by Both Model-Based and Model-Free Mechanisms.

机构信息

The Netherlands Institute for Neuroscience, Royal Netherlands Academy of Arts and Sciences, Amsterdam 1105BA, The Netherlands.

Department of Psychiatry, Amsterdam University Medical Centers, University of Amsterdam, Amsterdam 1105AZ, The Netherlands.

出版信息

eNeuro. 2024 Jul 3;11(7). doi: 10.1523/ENEURO.0223-24.2024. Print 2024 Jul.

Abstract

The magnitude of dopamine signals elicited by rewarding events and their predictors is updated when reward value changes. It is actively debated how readily these dopamine signals adapt and whether adaptation aligns with model-free or model-based reinforcement-learning principles. To investigate this, we trained male rats in a pavlovian-conditioning paradigm and measured dopamine release in the nucleus accumbens core in response to food reward (unconditioned stimulus) and reward-predictive conditioned stimuli (CS), both before and after reward devaluation, induced via either sensory-specific or nonspecific satiety. We demonstrate that (1) such devaluation reduces CS-induced dopamine release rapidly, without additional pairing of CS with devalued reward and irrespective of whether the devaluation was sensory-specific or nonspecific. In contrast, (2) reward devaluation did not decrease food reward-induced dopamine release. Surprisingly, (3) postdevaluation reconditioning, by additional pairing of CS with devalued reward, rapidly reinstated CS-induced dopamine signals to predevaluation levels. Taken together, we identify distinct, divergent adaptations in dopamine-signal magnitude when reward value is decreased: CS dopamine diminishes but reinstates fast, whereas reward dopamine is resistant to change. This implies that, respective to abovementioned findings, (1) CS dopamine may be governed by a model-based mechanism and (2) reward dopamine by a model-free one, where (3) the latter may contribute to swift reinstatement of the former. However, changes in CS dopamine were not selective for sensory specificity of reward devaluation, which is inconsistent with model-based processes. Thus, mesolimbic dopamine signaling incorporates both model-free and model-based mechanisms and is not exclusively governed by either.

摘要

当奖励价值发生变化时,奖赏事件引起的多巴胺信号的幅度及其预测因子会得到更新。目前积极争论的是,这些多巴胺信号是否容易适应,以及适应是否与无模型或基于模型的强化学习原则一致。为了研究这个问题,我们在条件反射范式中训练雄性大鼠,并在奖励贬值之前和之后,测量伏隔核核心中多巴胺的释放,以响应食物奖励(无条件刺激)和奖励预测条件刺激(CS),奖励贬值是通过感官特异性或非特异性饱腹感来诱导的。我们证明了:(1)这种贬值会迅速降低 CS 引起的多巴胺释放,而无需 CS 与贬值奖励的额外配对,也与贬值是否具有感官特异性或非特异性无关。相比之下,(2)奖励贬值不会降低食物奖励引起的多巴胺释放。令人惊讶的是,(3)通过 CS 与贬值奖励的额外配对进行的事后再条件作用,会迅速将 CS 引起的多巴胺信号恢复到贬值前的水平。总之,当奖励价值降低时,我们发现多巴胺信号幅度存在明显的、不同的适应:CS 多巴胺减少但快速恢复,而奖励多巴胺则不易变化。这意味着,相对于上述发现,(1)CS 多巴胺可能受到基于模型的机制的控制,(2)奖励多巴胺受到无模型的机制的控制,其中(3)后者可能有助于前者的快速恢复。然而,CS 多巴胺的变化并不针对奖励贬值的感官特异性,这与基于模型的过程不一致。因此,中脑边缘多巴胺信号包含无模型和基于模型的机制,而不是仅由其中之一控制。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/e931/11223458/5585f7df44d0/eneuro-11-ENEURO.0223-24.2024-g001.jpg

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验