Takahashi Yuji K, Batchelor Hannah M, Liu Bing, Khanna Akash, Morales Marisela, Schoenbaum Geoffrey
Intramural Research Program of the National Institute on Drug Abuse, NIH, Baltimore, MD 21224, USA.
Intramural Research Program of the National Institute on Drug Abuse, NIH, Baltimore, MD 21224, USA.
Neuron. 2017 Sep 13;95(6):1395-1405.e3. doi: 10.1016/j.neuron.2017.08.025.
Midbrain dopamine neurons have been proposed to signal prediction errors as defined in model-free reinforcement learning algorithms. While these algorithms have been extremely powerful in interpreting dopamine activity, these models do not register any error unless there is a difference between the value of what is predicted and what is received. Yet learning often occurs in response to changes in the unique features that characterize what is received, sometimes with no change in its value at all. Here, we show that classic error-signaling dopamine neurons also respond to changes in value-neutral sensory features of an expected reward. This suggests that dopamine neurons have access to a wider variety of information than contemplated by the models currently used to interpret their activity and that, while their firing may conform to predictions of these models in some cases, they are not restricted to signaling errors in the prediction of value.
中脑多巴胺神经元被认为可根据无模型强化学习算法中定义的预测误差发出信号。虽然这些算法在解释多巴胺活动方面极为强大,但除非预测值与实际接收值之间存在差异,否则这些模型不会记录任何误差。然而,学习通常是对所接收事物独特特征的变化做出反应,有时其价值根本没有变化。在这里,我们表明经典的误差信号多巴胺神经元也会对预期奖励的价值中性感官特征的变化做出反应。这表明多巴胺神经元能够获取比目前用于解释其活动的模型所考虑的更广泛的信息,并且虽然它们的放电在某些情况下可能符合这些模型的预测,但它们并不局限于在价值预测中发出误差信号。