Physical and Health Education, Graduate School of Education, the University of Tokyo, Tokyo, 113-0033, Japan.
Department of Biological Sciences, Graduate School of Science, the University of Tokyo, Tokyo, 113-0033, Japan.
eNeuro. 2018 Feb 21;5(1). doi: 10.1523/ENEURO.0021-18.2018. eCollection 2018 Jan-Feb.
Dopamine has been suggested to be crucially involved in effort-related choices. Key findings are that dopamine depletion (i) changed preference for a high-cost, large-reward option to a low-cost, small-reward option, (ii) but not when the large-reward option was also low-cost or the small-reward option gave no reward, (iii) while increasing the latency in all the cases but only transiently, and (iv) that antagonism of either dopamine D1 or D2 receptors also specifically impaired selection of the high-cost, large-reward option. The underlying neural circuit mechanisms remain unclear. Here we show that findings i-iii can be explained by the dopaminergic representation of temporal-difference reward-prediction error (TD-RPE), whose mechanisms have now become clarified, if (1) the synaptic strengths storing the values of actions mildly decay in time and (2) the obtained-reward-representing excitatory input to dopamine neurons increases after dopamine depletion. The former is potentially caused by background neural activity-induced weak synaptic plasticity, and the latter is assumed to occur through post-depletion increase of neural activity in the pedunculopontine nucleus, where neurons representing obtained reward exist and presumably send excitatory projections to dopamine neurons. We further show that finding iv, which is nontrivial given the suggested distinct functions of the D1 and D2 corticostriatal pathways, can also be explained if we additionally assume a proposed mechanism of TD-RPE calculation, in which the D1 and D2 pathways encode the values of actions with a temporal difference. These results suggest a possible circuit mechanism for the involvements of dopamine in effort-related choices and, simultaneously, provide implications for the mechanisms of TD-RPE calculation.
多巴胺被认为在与努力相关的选择中起着至关重要的作用。主要发现包括：（i）多巴胺耗竭使人们更喜欢高成本、大回报的选项而不是低成本、小回报的选项；（ii）但当大回报选项的成本也较低或小回报选项没有回报时，这种情况不会发生；（iii）尽管所有情况下的潜伏期都会增加，但只是暂时的；（iv）多巴胺 D1 或 D2 受体的拮抗剂也会特异性地损害对高成本、大回报选项的选择。其潜在的神经回路机制尚不清楚。在这里，我们表明，发现 i-iii 可以用多巴胺能对时间差奖励预测误差（TD-RPE）的表示来解释，如果（1）存储动作值的突触强度随时间轻微衰减，（2）多巴胺耗竭后，获得奖励所代表的兴奋性输入增加。前者可能是由背景神经活动引起的弱突触可塑性引起的，后者假设是通过多巴胺神经元所在的脑桥被盖核（pedunculopontine nucleus）中的神经活动在耗竭后增加而发生的，该核中存在代表获得奖励的神经元，并可能向多巴胺神经元发送兴奋性投射。我们进一步表明，发现 iv 也可以用 TD-RPE 计算的建议机制来解释，这是一个具有挑战性的发现，因为 D1 和 D2 皮质纹状体通路具有不同的功能。如果我们进一步假设 TD-RPE 计算的建议机制，即 D1 和 D2 通路以时间差编码动作的价值，那么这也是可以解释的。这些结果表明了多巴胺在与努力相关的选择中的可能的回路机制，同时为 TD-RPE 计算的机制提供了启示。
