Neuroscience Research Unit, Department of Psychology, University of Sheffield, Sheffield S10 2TP, UK.
Neuroscience. 2011 Dec 15;198:138-51. doi: 10.1016/j.neuroscience.2011.07.060. Epub 2011 Jul 29.
Multifunctional agents with limited motor resources must decide what actions will best ensure their survival. Moreover, given that in an unpredictable world things don't always work out, considerable advantage is to be gained by learning from experience - instrumental behaviour that maximises reward and minimises punishment. In this review we will argue that the re-entrant looped architecture of the basal ganglia represents biological solutions to these fundamental behavioural problems of selection and reinforcement. A potential solution to the selection problem is provided for by selective disinhibition within the parallel loop architecture that connects the basal ganglia with external neural structures. The relay points within these loops permit the signals of a particular channel to be modified by external influences. In part, these influences have the capacity to modify overall selections so that the probability of re-selecting reinforced behaviours in the future is altered. This is the basic process of instrumental learning, which we suggest decomposes into two sub-problems for the agent: (i) learning which external events it causes to happen and learning precisely what it is doing that is causal; and (ii) having determined agency and discovered novel action-outcome routines, how best to exploit this knowledge to maximise future reward acquisitions. Considerations of connectional architecture and signal timing suggest that the short-latency, sensory-evoked dopamine response, which can modulate the re-entrant loop structure within the basal ganglia, is ideally suited to reinforce the determination of agency and the discovery of novel actions. Alternatively, recent studies showing that presence or absence of reward can selectively modulate the magnitude of signals in structures providing input signals to the basal ganglia, offer an alternative mechanism for biasing selection within the re-entrant loop architecture. We suggest that this mechanism may be better suited to ensure the prioritisation of inputs associated with reward.
具有有限运动资源的多功能剂必须决定哪些行动将最能确保它们的生存。此外，由于在一个不可预测的世界中，事情并不总是按计划进行，因此通过经验学习（最大限度地提高奖励和最小化惩罚的工具行为）可以获得相当大的优势。在这篇综述中，我们将论证基底神经节的折返环结构代表了对选择和强化这些基本行为问题的生物学解决方案。平行回路结构中选择性去抑制为选择问题提供了潜在的解决方案，该结构将基底神经节与外部神经结构连接起来。这些回路中的中继点允许特定通道的信号被外部影响修改。在某种程度上，这些影响有能力改变整体选择，以便将来重新选择强化行为的概率发生改变。这是工具学习的基本过程，我们建议将其分解为代理的两个子问题：（i）学习它导致外部事件发生的原因，以及确切地了解它正在做什么是因果关系；（ii）确定代理机构并发现新的行动-结果例程后，如何最好地利用这些知识来最大限度地提高未来的奖励获取。连接结构和信号定时的考虑表明，短潜伏期、感官诱发的多巴胺反应可以调节基底神经节内的折返环结构，非常适合强化代理机构的确定和新动作的发现。或者，最近的研究表明，奖励的存在或不存在可以选择性地调节向基底神经节提供输入信号的结构中的信号幅度，为折返环结构内的选择提供了另一种偏向机制。我们认为，这种机制可能更适合确保与奖励相关的输入的优先级。
