Zhu Qinghua, Han Fang, Yuan Yuanyuan, Shen Luyi
College of Information Science and Technology, Donghua University, Shanghai, 201620 China.
College of Science, Donghua University, Shanghai, 201620 China.
Cogn Neurodyn. 2024 Oct;18(5):2127-2144. doi: 10.1007/s11571-023-10046-0. Epub 2023 Dec 23.
The basal ganglia (BG) plays a key role in action selection. Physiological experiments have suggested that the reciprocal interaction between tonically active neurons (TANs) and dopamine (DA) is closely related to reward-based behaviors. However, the functional role of TAN-DA interaction in action selection remains unclear. In this study, a cortico-BG model including TAN-DA interaction mechanism is developed to explore the action selection mechanism of BG. The results show that in the default case, direct, indirect, and hyperdirect pathways are responsible for promoting, suppressing, and stopping the formation of stimulus-action associations, respectively. In the case of reinforcement learning, a single rewarded action is selected according to the combination of the TAN-DA dependent reinforcement mechanism and Hebbian mechanism with a gradual transfer from the former to the latter. Besides, a longer exploratory phase occurs when switching the reward to a new action because additional trials are required to overcome the habituation previously induced by the Hebbian mechanism. In the Parkinsonian state, the reinforcement mechanism is disrupted, and the resting tremor occurs due to dopamine deficiency. Although the model's performance significantly improves due to the levodopa treatment, it is still inferior to the healthy state. This phenomenon is consistent with the experimental results and is explained theoretically via the TAN pause duration and phasic DA release. Furthermore, the model's performances in multi-action selection further verify the rationality of the TAN-DA-dependent reinforcement mechanism. Our work provides a more complete framework for studying the action selection mechanism of basal ganglia.
基底神经节（BG）在动作选择中起关键作用。生理学实验表明，持续活动神经元（TANs）与多巴胺（DA）之间的相互作用与基于奖励的行为密切相关。然而，TAN-DA相互作用在动作选择中的功能作用仍不清楚。在本研究中，开发了一个包含TAN-DA相互作用机制的皮质-基底神经节模型，以探索基底神经节的动作选择机制。结果表明，在默认情况下，直接通路、间接通路和超直接通路分别负责促进、抑制和停止刺激-动作关联的形成。在强化学习的情况下，根据TAN-DA依赖的强化机制和赫布机制的组合选择单个奖励动作，并从前一种机制向后一种机制逐渐转变。此外，当将奖励切换到新动作时，会出现更长的探索阶段，因为需要额外的试验来克服先前由赫布机制引起的习惯化。在帕金森状态下，强化机制被破坏，由于多巴胺缺乏而出现静止性震颤。尽管左旋多巴治疗显著改善了模型的性能，但仍低于健康状态。这一现象与实验结果一致，并通过TAN暂停持续时间和阶段性多巴胺释放从理论上进行了解释。此外，模型在多动作选择中的性能进一步验证了TAN-DA依赖强化机制的合理性。我们的工作为研究基底神经节的动作选择机制提供了一个更完整的框架。
