Kahnt Thorsten, Park Soyoung Q, Cohen Michael X, Beck Anne, Heinz Andreas, Wrase Jana
Department of Psychiatry and Psychotherapy, Charité-Universitätsmedizin Berlin (Charité Campus Mitte), Berlin, Germany.
J Cogn Neurosci. 2009 Jul;21(7):1332-45. doi: 10.1162/jocn.2009.21092.
It has been suggested that the target areas of dopaminergic midbrain neurons, the dorsal (DS) and ventral striatum (VS), are differently involved in reinforcement learning especially as actor and critic. Whereas the critic learns to predict rewards, the actor maintains action values to guide future decisions. The different midbrain connections to the DS and the VS seem to play a critical role in this functional distinction. Here, subjects performed a dynamic, reward-based decision-making task during fMRI acquisition. A computational model of reinforcement learning was used to estimate the different effects of positive and negative reinforcements on future decisions for each subject individually. We found that activity in both the DS and the VS correlated with reward prediction errors. Using functional connectivity, we show that the DS and the VS are differentially connected to different midbrain regions (possibly corresponding to the substantia nigra [SN] and the ventral tegmental area [VTA], respectively). However, only functional connectivity between the DS and the putative SN predicted the impact of different reinforcement types on future behavior. These results suggest that connections between the putative SN and the DS are critical for modulating action values in the DS according to both positive and negative reinforcements to guide future decision making.
有人提出,多巴胺能中脑神经元的目标区域,即背侧纹状体(DS)和腹侧纹状体(VS),在强化学习中有着不同的参与方式,特别是作为行动者和评判者。评判者学习预测奖励,而行动者维持行动价值以指导未来决策。中脑与DS和VS的不同连接似乎在这种功能区分中起着关键作用。在此,受试者在功能磁共振成像(fMRI)采集过程中执行了一项基于奖励的动态决策任务。使用强化学习的计算模型来分别估计正向和负向强化对每个受试者未来决策的不同影响。我们发现DS和VS中的活动都与奖励预测误差相关。通过功能连接性,我们表明DS和VS与不同的中脑区域有不同的连接(可能分别对应黑质[SN]和腹侧被盖区[VTA])。然而,只有DS与假定的SN之间的功能连接预测了不同强化类型对未来行为的影响。这些结果表明,假定的SN与DS之间的连接对于根据正向和负向强化来调节DS中的行动价值以指导未来决策至关重要。