Zhu Tianyao, Areshenkoff Corson N, De Brouwer Anouk J, Nashed Joseph Y, Flanagan J Randall, Gallivan Jason P
Center for Neuroscience Studies, Queen's University, Kingston, Ontario, Canada.
Department of Psychology, Queen's University, Kingston, Ontario, Canada.
J Neurosci. 2025 Mar 18;45(18). doi: 10.1523/JNEUROSCI.2158-24.2025.
How the brain learns new motor commands through reinforcement involves distributed neural circuits beyond known frontal-striatal pathways, yet a comprehensive understanding of this broader neural architecture remains elusive. Here, using human functional MRI ( = 46, 27 females) and manifold learning techniques, we identified a low-dimensional neural space that captured the dynamic changes in whole-brain functional organization during a reward-based trajectory learning task. By quantifying participants' learning rates through an Actor-Critic model, we discovered that periods of accelerated learning were characterized by significant manifold contractions across multiple brain regions, including areas of limbic and hippocampal cortex, as well as the cerebellum. This contraction reflected enhanced network integration, with notably stronger connectivity between several of these regions and the sensorimotor cerebellum correlating with higher learning rates. These findings challenge the traditional view of the cerebellum as solely involved in error-based learning, supporting the emerging view that it coordinates with other brain regions during reinforcement learning. This study reveals how distributed brain systems, including the cerebellum and hippocampus, alter their functional connectivity to support motor learning through reinforcement. Using advanced manifold learning techniques on functional MRI data, we examined changes in regional connectivity during reward-based learning and their relationship to learning rate. For several brain regions, we found that periods of heightened learning were associated with increased cerebellar connectivity, suggesting a key role for the cerebellum in reward-based motor learning. These findings challenge the traditional view of the cerebellum as solely involved in supervised (error-based) learning and add to a growing rodent literature supporting a role for cerebellar circuits in reward-driven learning.
大脑如何通过强化学习来掌握新的运动指令，涉及到已知额叶-纹状体通路之外的分布式神经回路，但对这种更广泛的神经结构的全面理解仍然难以捉摸。在这里，我们使用人类功能性磁共振成像（ = 46，27名女性）和流形学习技术，识别出一个低维神经空间，该空间捕捉了基于奖励的轨迹学习任务期间全脑功能组织的动态变化。通过一个演员-评论家模型量化参与者的学习率，我们发现学习加速期的特征是多个脑区出现显著的流形收缩，包括边缘和海马皮层区域以及小脑。这种收缩反映了网络整合的增强，其中这些区域中的几个与感觉运动小脑之间更强的连接与更高的学习率相关。这些发现挑战了传统观点，即小脑仅参与基于误差的学习，支持了一种新出现的观点，即它在强化学习过程中与其他脑区协同作用。这项研究揭示了包括小脑和海马体在内的分布式脑系统如何改变其功能连接，以通过强化来支持运动学习。我们使用先进的流形学习技术处理功能性磁共振成像数据，研究了基于奖励的学习过程中区域连接的变化及其与学习率的关系。对于几个脑区，我们发现学习增强期与小脑连接增加有关，这表明小脑在基于奖励的运动学习中起关键作用。这些发现挑战了传统观点，即小脑仅参与监督（基于误差）学习，并补充了越来越多的啮齿动物文献，支持小脑回路在奖励驱动学习中的作用。
