Department of Experimental Psychology, Ghent University.
Department of Psychological and Brain Sciences, Indiana University.
Top Cogn Sci. 2019 Jan;11(1):119-135. doi: 10.1111/tops.12307. Epub 2017 Nov 13.
In the past two decades, reinforcement learning (RL) has become a popular framework for understanding brain function. A key component of RL models, prediction error, has been associated with neural signals throughout the brain, including subcortical nuclei, primary sensory cortices, and prefrontal cortex. Depending on the location in which activity is observed, the functional interpretation of prediction error may change: Prediction errors may reflect a discrepancy in the anticipated and actual value of reward, a signal indicating the salience or novelty of a stimulus, and many other interpretations. Anterior cingulate cortex (ACC) has long been recognized as a region involved in processing behavioral error, and recent computational models of the region have expanded this interpretation to include a more general role for the region in predicting likely events, broadly construed, and signaling deviations between expected and observed events. Ongoing modeling work investigating the interaction between ACC and additional regions involved in cognitive control suggests an even broader role for cingulate in computing a hierarchically structured surprise signal critical for learning models of the environment. The result is a predictive coding model of the frontal lobes, suggesting that predictive coding may be a unifying computational principle across the neocortex.
在过去的二十年中，强化学习（RL）已成为理解大脑功能的流行框架。RL 模型的一个关键组成部分，即预测误差，与整个大脑中的神经信号有关，包括皮质下核、初级感觉皮层和前额叶皮层。根据观察到的活动的位置，预测误差的功能解释可能会发生变化：预测误差可能反映了预期和实际奖励值之间的差异，或者是一个刺激的显著或新奇性的信号，以及许多其他解释。前扣带皮层（ACC）长期以来一直被认为是参与处理行为错误的区域，该区域的最近计算模型将这一解释扩展到包括该区域在更广泛地预测可能事件中的一般作用，广义上讲，以及在预期和观察到的事件之间发出偏差信号。正在进行的建模工作调查了 ACC 与参与认知控制的其他区域之间的相互作用，这表明扣带在计算对环境学习模型至关重要的分层结构的惊喜信号方面发挥着更广泛的作用。其结果是一个额叶的预测编码模型，表明预测编码可能是整个新皮层的统一计算原理。
