Bossaerts Peter
Department of Finance, Faculty of Business and Economics, The University of Melbourne, Melbourne, VIC, Australia.
The Florey Institute of Neuroscience and Mental Health, Parkville, VIC, Australia.
Front Integr Neurosci. 2018 Dec 4;12:61. doi: 10.3389/fnint.2018.00061. eCollection 2018.
Anterior insula (aIns) is thought to play a crucial role in rapid adaptation in an ever-changing environment. Mathematically, it is known to track risk and surprise. Modern theories of learning, however, assign a dominant role to prediction errors (PEs), not to risk and surprise. Risk and surprise only enter to the extent that they modulate the learning rate, in an attempt to approximate Bayesian learning. Even without such modulation, adaptation is still possible, albeit slow. Here, I propose a new theory of learning, reference-model based learning (RMBL), where risk and surprise are central, and PEs play a secondary, though still crucial, role. The primary goal is to bring outcomes in line with expectations in the reference model (RM). Learning is modulated by how large the PEs are relative to model anticipation, i.e., to surprise as defined by the RM. In a target location prediction task where participants were continuously required to adapt, choices appeared to be closer with to RMBL predictions than to Bayesian learning. aIns reaction to surprise was more acute in the more difficult treatment, consistent with its hypothesized role in metacognition. I discuss links with related theories, such as Active Inference, Actor-Critic Models and Reference-Model Based Adaptive Control.
前脑岛（aIns）被认为在不断变化的环境中快速适应方面起着关键作用。从数学角度来看，它能够追踪风险和意外情况。然而，现代学习理论将主要作用赋予了预测误差（PEs），而非风险和意外情况。风险和意外情况仅在它们调节学习率的程度上起作用，试图近似贝叶斯学习。即使没有这种调节，适应仍然是可能的，尽管会很缓慢。在此，我提出一种新的学习理论，即基于参考模型的学习（RMBL），其中风险和意外情况是核心，而预测误差起着次要但仍然至关重要的作用。主要目标是使结果与参考模型（RM）中的预期相符。学习是由预测误差相对于模型预期的大小来调节的，即相对于参考模型所定义的意外情况。在一个参与者被持续要求适应的目标位置预测任务中，选择似乎更接近基于参考模型的学习预测，而非贝叶斯学习。在前脑岛对意外情况的反应在更困难的处理中更为敏锐，这与其在元认知中假设的作用一致。我讨论了与相关理论的联系，如主动推理、演员 - 评论家模型和基于参考模型的自适应控制。
