Chater Nick
Division of Psychology and Language Sciences, Centre for Economic Learning and Social Evolution (ELSE), UCL, London, WC1E 6BT, United Kingdom.
Cognition. 2009 Dec;113(3):350-364. doi: 10.1016/j.cognition.2008.06.014. Epub 2008 Aug 22.
This special issue describes important recent developments in applying reinforcement learning models to capture neural and cognitive function. But reinforcement learning, as a theoretical framework, can apply at two very different levels of description: mechanistic and rational. Reinforcement learning is often viewed in mechanistic terms--as describing the operation of aspects of an agent's cognitive and neural machinery. Yet it can also be viewed as a rational level of description, specifically, as describing a class of methods for learning from experience, using minimal background knowledge. This paper considers how rational and mechanistic perspectives differ, and what types of evidence distinguish between them. Reinforcement learning research in the cognitive and brain sciences is often implicitly committed to the mechanistic interpretation. Here the opposite view is put forward: that accounts of reinforcement learning should apply at the rational level, unless there is strong evidence for a mechanistic interpretation. Implications of this viewpoint for reinforcement-based theories in the cognitive and brain sciences are discussed.
本期特刊描述了应用强化学习模型来捕捉神经和认知功能方面的重要近期进展。但是，强化学习作为一个理论框架，可以在两个非常不同的描述层面上应用：机制层面和理性层面。强化学习通常从机制角度来看待——即描述智能体认知和神经机制各方面的运作。然而，它也可以被视为一种理性层面的描述，具体而言，是描述一类利用最少背景知识从经验中学习的方法。本文探讨了理性视角与机制视角有何不同，以及哪些类型的证据可以区分它们。认知和脑科学中的强化学习研究常常隐含地倾向于机制性解释。这里提出相反的观点：除非有强有力的证据支持机制性解释，否则强化学习的解释应适用于理性层面。本文还讨论了这一观点对认知和脑科学中基于强化学习的理论的影响。
