Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, 6500 HB Nijmegen, the Netherlands,
Montreal Neurological Institute, McGill University, Montreal, QC H3A 2B4, Canada.
J Neurosci. 2014 Jun 4;34(23):7814-24. doi: 10.1523/JNEUROSCI.4063-13.2014.
A substantial subset of Parkinson's disease (PD) patients suffers from impulse control disorders (ICDs), which are side effects of dopaminergic medication. Dopamine plays a key role in reinforcement learning processes. One class of reinforcement learning models, known as the actor-critic model, suggests that two components are involved in these reinforcement learning processes: a critic, which estimates values of stimuli and calculates prediction errors, and an actor, which estimates values of potential actions. To understand the information processing mechanism underlying impulsive behavior, we investigated stimulus and action value learning from reward and punishment in four groups of participants: on-medication PD patients with ICD, on-medication PD patients without ICD, off-medication PD patients without ICD, and healthy controls. Analysis of responses suggested that participants used an actor-critic learning strategy and computed prediction errors based on stimulus values rather than action values. Quantitative model fits also revealed that an actor-critic model of the basal ganglia with different learning rates for positive and negative prediction errors best matched the choice data. Moreover, whereas ICDs were associated with model parameters related to stimulus valuation (critic), PD was associated with parameters related to action valuation (actor). Specifically, PD patients with ICD exhibited lower learning from negative prediction errors in the critic, resulting in an underestimation of adverse consequences associated with stimuli. These findings offer a specific neurocomputational account of the nature of compulsive behaviors induced by dopaminergic drugs.
相当一部分帕金森病 (PD) 患者患有冲动控制障碍 (ICD)，这是多巴胺能药物的副作用。多巴胺在强化学习过程中起着关键作用。一类强化学习模型，称为行为-评价模型，表明这些强化学习过程涉及两个组成部分：评价者，估计刺激的价值并计算预测误差；行为者，估计潜在动作的价值。为了了解冲动行为的信息处理机制，我们研究了来自奖励和惩罚的刺激和动作价值学习，参与者分为四组：患有 ICD 的药物治疗 PD 患者、无 ICD 的药物治疗 PD 患者、无 ICD 的停药 PD 患者和健康对照组。对反应的分析表明，参与者使用了行为-评价学习策略，并根据刺激值而不是动作值计算预测误差。定量模型拟合也表明，具有正、负预测误差不同学习率的基底神经节行为-评价模型最能匹配选择数据。此外，ICD 与与刺激估值（评价者）相关的模型参数相关，而 PD 与与动作估值（行为者）相关的参数相关。具体而言，患有 ICD 的 PD 患者在评价者中表现出对负预测误差的学习能力降低，导致对与刺激相关的不利后果的低估。这些发现为多巴胺能药物引起的强迫行为的本质提供了特定的神经计算解释。
