Hélie Sébastien, Fleischer Pierson J
Department of Psychological Sciences, Purdue University West Lafayette, IN, USA.
Front Comput Neurosci. 2016 Apr 29;10:40. doi: 10.3389/fncom.2016.00040. eCollection 2016.
The study of rhythms and oscillations in the brain is gaining attention. While it is unclear exactly what the role of oscillation, synchrony, and rhythm is, it appears increasingly likely that synchrony is related to normal and abnormal brain states and possibly cognition. In this article, we explore the relationship between basal ganglia (BG) synchrony and reinforcement learning. We simulate a biologically-realistic model of the striatum initially proposed by Ponzi and Wickens (2010) and enhance the model by adding plastic cortico-BG synapses that can be modified using reinforcement learning. The effect of reinforcement learning on striatal rhythmic activity is then explored, and disrupted using simulated deep brain stimulation (DBS). The stimulator injects current in the brain structure to which it is attached, which affects neuronal synchrony. The results show that training the model without DBS yields a high accuracy in the learning task and reduced the number of active neurons in the striatum, along with an increased firing periodicity and a decreased firing synchrony between neurons in the same assembly. In addition, a spectral decomposition shows a stronger signal for correct trials than incorrect trials in high frequency bands. If the DBS is ON during the training phase, but not the test phase, the amount of learning in the model is reduced, along with firing periodicity. Similar to when the DBS is OFF, spectral decomposition shows a stronger signal for correct trials than for incorrect trials in high frequency domains, but this phenoemenon happens in higher frequency bands than when the DBS is OFF. Synchrony between the neurons is not affected. Finally, the results show that turning the DBS ON at test increases both firing periodicity and striatal synchrony, and spectral decomposition of the signal show that neural activity synchronizes with the DBS fundamental frequency (and its harmonics). Turning the DBS ON during the test phase results in chance performance regardless of whether the DBS was ON or OFF during training. We conclude that reinforcement learning is related to firing periodicity, and a stronger signal for correct trials when compared to incorrect trials in high frequency bands.
对大脑节律和振荡的研究正日益受到关注。虽然目前尚不清楚振荡、同步性和节律的确切作用，但同步性似乎越来越有可能与正常和异常的脑状态以及认知相关。在本文中，我们探讨基底神经节（BG）同步性与强化学习之间的关系。我们模拟了庞兹和威肯斯（2010年）最初提出的纹状体生物现实模型，并通过添加可使用强化学习进行修改的可塑性皮质 - BG突触来增强该模型。然后研究强化学习对纹状体节律活动的影响，并使用模拟深部脑刺激（DBS）对其进行干扰。刺激器向与其相连的脑结构中注入电流，这会影响神经元同步性。结果表明，在无DBS的情况下训练模型，在学习任务中能产生高精度，减少纹状体中活跃神经元的数量，同时增加放电周期性并降低同一组件中神经元之间的放电同步性。此外，频谱分解显示在高频带中正确试验的信号比错误试验的信号更强。如果在训练阶段开启DBS，但在测试阶段关闭，模型中的学习量会减少，同时放电周期性也会降低。与DBS关闭时类似，频谱分解显示在高频域中正确试验的信号比错误试验的信号更强，但这种现象发生的频段比DBS关闭时更高。神经元之间的同步性不受影响。最后，结果表明在测试时开启DBS会增加放电周期性和纹状体同步性，信号的频谱分解表明神经活动与DBS基频（及其谐波）同步。在测试阶段开启DBS会导致随机表现，无论训练期间DBS是开启还是关闭。我们得出结论，强化学习与放电周期性相关，并且在高频带中正确试验比错误试验有更强的信号。
