Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London WC1B 5EH, United Kingdom;
Department of Management/MAPP, Aarhus University, Aarhus 8210, Denmark.
Proc Natl Acad Sci U S A. 2020 Feb 11;117(6):3291-3300. doi: 10.1073/pnas.1911348117. Epub 2020 Jan 24.
Uncertainty plays a critical role in reinforcement learning and decision making. However, exactly how it influences behavior remains unclear. Multiarmed-bandit tasks offer an ideal test bed, since computational tools such as approximate Kalman filters can closely characterize the interplay between trial-by-trial values, uncertainty, learning, and choice. To gain additional insight into learning and choice processes, we obtained data from subjects' overt allocation of gaze. The estimated value and estimation uncertainty of options influenced what subjects looked at before choosing; these same quantities also influenced choice, as additionally did fixation itself. A momentary measure of uncertainty in the form of absolute prediction errors determined how long participants looked at the obtained outcomes. These findings affirm the importance of uncertainty in multiple facets of behavior and help delineate its effects on decision making.
不确定性在强化学习和决策中起着关键作用。然而，它究竟如何影响行为尚不清楚。多臂赌博机任务提供了一个理想的测试平台，因为近似卡尔曼滤波器等计算工具可以很好地描述试次间价值、不确定性、学习和选择之间的相互作用。为了更深入地了解学习和选择过程，我们从被试者的注视分配中获得了数据。选项的估计值和估计不确定性影响了被试者在选择前看什么；这些相同的量也影响了选择，因为注视本身也是如此。以绝对预测误差形式表示的瞬间不确定性衡量标准决定了参与者观察所获得结果的时间长短。这些发现肯定了不确定性在行为的多个方面的重要性，并帮助描绘了它对决策的影响。
