Zhu Changbo, Zhou Ke, Tang Yandong, Tang Fengzhen, Si Bailu
State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, 110016 Liaoning China.
University of Chinese Academy of Sciences, Beijing, 100049 China.
Cogn Neurodyn. 2024 Dec;18(6):4009-4031. doi: 10.1007/s11571-024-10128-7. Epub 2024 Oct 21.
Adaptive mechanisms of learning models play critical roles in interpreting adaptive behavior of humans and animals. Different learning models, varying from Bayesian models, deep learning or regression models to reward-based reinforcement learning models, adopt similar update rules. These update rules can be reduced to the same generalized mathematical form: the Rescorla-Wagner equation. In this paper, we construct a hierarchical Bayesian model with an adaptive learning rate for inferring a hidden probability in a dynamical binary environment, and analysis the adaptive behavior of the model on synthetic data. The update rule of the model state turns out to be an extension of the Rescorla-Wagner equation. The adaptive learning rate is modulated by beliefs and environment uncertainty. Our results underscore adaptive learning rate as mechanistic component in efficient and accurate inference, as well as the signature of information processing in adaptive machine learning models.
学习模型的自适应机制在解释人类和动物的适应性行为方面起着关键作用。不同的学习模型，从贝叶斯模型、深度学习或回归模型到基于奖励的强化学习模型，都采用类似的更新规则。这些更新规则可以简化为相同的广义数学形式：雷斯克拉-瓦格纳方程。在本文中，我们构建了一个具有自适应学习率的分层贝叶斯模型，用于推断动态二元环境中的隐藏概率，并分析该模型在合成数据上的自适应行为。结果表明，模型状态的更新规则是雷斯克拉-瓦格纳方程的一种扩展。自适应学习率由信念和环境不确定性调节。我们的结果强调了自适应学习率作为高效准确推理中的机制组成部分，以及自适应机器学习模型中信息处理的特征。
