Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, United States of America.
PLoS Comput Biol. 2020 Jul 1;16(7):e1007963. doi: 10.1371/journal.pcbi.1007963. eCollection 2020 Jul.
Sound principles of statistical inference dictate that uncertainty shapes learning. In this work, we revisit the question of learning in volatile environments, in which both the first and second-order statistics of observations dynamically evolve over time. We propose a new model, the volatile Kalman filter (VKF), which is based on a tractable state-space model of uncertainty and extends the Kalman filter algorithm to volatile environments. The proposed model is algorithmically simple and encompasses the Kalman filter as a special case. Specifically, in addition to the error-correcting rule of Kalman filter for learning observations, the VKF learns volatility according to a second error-correcting rule. These dual updates echo and contextualize classical psychological models of learning, in particular hybrid accounts of Pearce-Hall and Rescorla-Wagner. At the computational level, compared with existing models, the VKF gives up some flexibility in the generative model to enable a more faithful approximation to exact inference. When fit to empirical data, the VKF is better behaved than alternatives and better captures human choice data in two independent datasets of probabilistic learning tasks. The proposed model provides a coherent account of learning in stable or volatile environments and has implications for decision neuroscience research.
稳健的统计推断原则表明，不确定性会影响学习。在这项工作中，我们重新探讨了在动态变化的环境中学习的问题，在这种环境中，观测的一阶和二阶统计数据随时间不断演变。我们提出了一种新的模型，即易变卡尔曼滤波器（VKF），它基于一种可处理的不确定性状态空间模型，并将卡尔曼滤波算法扩展到易变环境中。所提出的模型算法简单，并包含了卡尔曼滤波器作为一个特例。具体来说，除了卡尔曼滤波器用于学习观测的纠错规则外，VKF 根据第二个纠错规则来学习波动性。这些双重更新反映并将经典的学习心理模型语境化，特别是 Pearce-Hall 和 Rescorla-Wagner 的混合模型。在计算层面上，与现有模型相比，VKF 在生成模型中放弃了一些灵活性，以更准确地逼近精确推断。当拟合经验数据时，VKF 比其他模型表现更好，并且在两个独立的概率学习任务数据集的人类选择数据中更好地捕捉到了人类的选择。所提出的模型为稳定或易变环境中的学习提供了一个连贯的解释，并对决策神经科学研究具有重要意义。
