Portes Jacob P, Schmid Christian, Murray James M
Center for Theoretical Neuroscience, Columbia University.
Institute of Neuroscience, University of Oregon.
Adv Neural Inf Process Syst. 2022 Dec;35:25937-25950.
Despite extensive theoretical work on biologically plausible learning rules, clear evidence about whether and how such rules are implemented in the brain has been difficult to obtain. We consider biologically plausible supervised- and reinforcement-learning rules and ask whether changes in network activity during learning can be used to determine which learning rule is being used. Supervised learning requires a credit-assignment model estimating the mapping from neural activity to behavior, and, in a biological organism, this model will inevitably be an imperfect approximation of the ideal mapping, leading to a bias in the direction of the weight updates relative to the true gradient. Reinforcement learning, on the other hand, requires no credit-assignment model and tends to make weight updates following the true gradient direction. We derive a metric to distinguish between learning rules by observing changes in the network activity during learning, given that the mapping from brain to behavior is known by the experimenter. Because brain-machine interface (BMI) experiments allow for precise knowledge of this mapping, we model a cursor-control BMI task using recurrent neural networks, showing that learning rules can be distinguished in simulated experiments using only observations that a neuroscience experimenter would plausibly have access to.
尽管在具有生物学合理性的学习规则方面已经开展了大量理论研究工作，但关于此类规则是否以及如何在大脑中得以实现的明确证据却一直难以获得。我们考虑具有生物学合理性的监督学习和强化学习规则，并探究学习过程中网络活动的变化是否可用于确定正在使用的是哪种学习规则。监督学习需要一个信用分配模型来估计从神经活动到行为的映射，而在生物有机体中，该模型不可避免地是理想映射的不完美近似，从而导致相对于真实梯度的权重更新方向出现偏差。另一方面，强化学习不需要信用分配模型，并且倾向于沿着真实梯度方向进行权重更新。鉴于实验者已知从大脑到行为的映射，我们通过观察学习过程中网络活动的变化来推导一种区分学习规则的度量。由于脑机接口（BMI）实验能够精确了解这种映射，我们使用循环神经网络对光标控制BMI任务进行建模，结果表明，在模拟实验中，仅使用神经科学实验者可能获取的观察结果就能区分学习规则。
