School of Psychology, University of Nottingham, Nottingham, UK.
Zurich Center for Neuroeconomics, Department of Economics, University of Zurich, Zurich, Switzerland.
Nat Hum Behav. 2023 Jun;7(6):956-969. doi: 10.1038/s41562-023-01573-1. Epub 2023 Apr 3.
A standard assumption in neuroscience is that low-effort model-free learning is automatic and continuously used, whereas more complex model-based strategies are only used when the rewards they generate are worth the additional effort. We present evidence refuting this assumption. First, we demonstrate flaws in previous reports of combined model-free and model-based reward prediction errors in the ventral striatum that probably led to spurious results. More appropriate analyses yield no evidence of model-free prediction errors in this region. Second, we find that task instructions generating more correct model-based behaviour reduce rather than increase mental effort. This is inconsistent with cost-benefit arbitration between model-based and model-free strategies. Together, our data indicate that model-free learning may not be automatic. Instead, humans can reduce mental effort by using a model-based strategy alone rather than arbitrating between multiple strategies. Our results call for re-evaluation of the assumptions in influential theories of learning and decision-making.
神经科学的一个标准假设是，低努力的无模型学习是自动的，并持续使用，而更复杂的基于模型的策略仅在它们产生的奖励值得额外努力时使用。我们提出了反驳这一假设的证据。首先，我们证明了以前关于腹侧纹状体中结合的无模型和基于模型的奖励预测误差的报告存在缺陷，这可能导致了虚假的结果。更合适的分析没有在这个区域发现无模型预测误差的证据。其次，我们发现，生成更正确基于模型行为的任务指令会减少而不是增加心理努力。这与基于模型和无模型策略之间的成本效益仲裁不一致。总之，我们的数据表明，无模型学习可能不是自动的。相反，人类可以通过仅使用基于模型的策略来减少心理努力，而不是在多种策略之间进行仲裁。我们的结果呼吁重新评估学习和决策的有影响力的理论中的假设。
