Department of Experimental Psychology, Ghent University, Ghent, Belgium.
Department of Experimental Psychology, Ghent University, Ghent, Belgium.
Neurosci Biobehav Rev. 2024 May;160:105623. doi: 10.1016/j.neubiorev.2024.105623. Epub 2024 Mar 13.
Foraging is a natural behavior that involves making sequential decisions to maximize rewards while minimizing the costs incurred when doing so. The prevalence of foraging across species suggests that a common brain computation underlies its implementation. Although anterior cingulate cortex is believed to contribute to foraging behavior, its specific role has been contentious, with predominant theories arguing either that it encodes environmental value or choice difficulty. Additionally, recent attempts to characterize foraging have taken place within the reinforcement learning framework, with increasingly complex models scaling with task complexity. Here we review reinforcement learning foraging models, highlighting the hierarchical structure of many foraging problems. We extend this literature by proposing that ACC guides foraging according to principles of model-based hierarchical reinforcement learning. This idea holds that ACC function is organized hierarchically along a rostral-caudal gradient, with rostral structures monitoring the status and completion of high-level task goals (like finding food), and midcingulate structures overseeing the execution of task options (subgoals, like harvesting fruit) and lower-level actions (such as grabbing an apple).
觅食是一种自然行为，涉及到做出连续的决策，以最大化奖励，同时最小化这样做所带来的成本。觅食在物种中的普遍存在表明，其实施背后存在着一种共同的大脑计算。尽管前扣带皮层被认为对觅食行为有贡献，但它的具体作用一直存在争议，主要理论要么认为它编码了环境价值，要么认为它编码了选择难度。此外，最近在强化学习框架内对觅食进行了特征描述，越来越复杂的模型与任务复杂性成比例地扩展。在这里，我们回顾了强化学习觅食模型，强调了许多觅食问题的分层结构。我们通过提出 ACC 根据基于模型的分层强化学习的原则来指导觅食来扩展这一文献。这个想法认为，ACC 功能沿着一个头侧尾侧的梯度进行分层组织，头侧结构监测高级任务目标的状态和完成情况（如寻找食物），而中扣带结构监督任务选项（子目标，如收获水果）和较低层次的动作（如抓取苹果）的执行情况。
