Department of Bio and Brain Engineering, Korea Advanced Institute of Science Technology (KAIST), Daejeon, 34141, Republic of Korea.
Program of Brain and Cognitive Engineering, KAIST, Daejeon, 34141, Republic of Korea.
Nat Commun. 2019 Dec 16;10(1):5738. doi: 10.1038/s41467-019-13632-1.
It has previously been shown that the relative reliability of model-based and model-free reinforcement-learning (RL) systems plays a role in the allocation of behavioral control between them. However, the role of task complexity in the arbitration between these two strategies remains largely unknown. Here, using a combination of novel task design, computational modelling, and model-based fMRI analysis, we examined the role of task complexity alongside state-space uncertainty in the arbitration process. Participants tended to increase model-based RL control in response to increasing task complexity. However, they resorted to model-free RL when both uncertainty and task complexity were high, suggesting that these two variables interact during the arbitration process. Computational fMRI revealed that task complexity interacts with neural representations of the reliability of the two systems in the inferior prefrontal cortex.
先前的研究表明,基于模型和无模型强化学习(RL)系统的相对可靠性在它们之间的行为控制分配中起着作用。然而,任务复杂性在这两种策略之间的仲裁中的作用在很大程度上仍然未知。在这里,我们使用新颖的任务设计、计算建模和基于模型的 fMRI 分析的组合,研究了任务复杂性以及状态空间不确定性在仲裁过程中的作用。参与者倾向于随着任务复杂性的增加而增加基于模型的 RL 控制。然而,当不确定性和任务复杂性都很高时,他们求助于无模型 RL,这表明这两个变量在仲裁过程中相互作用。计算 fMRI 显示,任务复杂性与下前额叶皮层中两个系统可靠性的神经表示相互作用。