Suppr超能文献

基于模型的树搜索中的证据整合。

Evidence integration in model-based tree search.

作者信息

Solway Alec, Botvinick Matthew M

机构信息

Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544;

Princeton Neuroscience Institute, Princeton University, Princeton, NJ 08544; Department of Psychology, Princeton University, Princeton, NJ 08544; Google DeepMind, London EC4A 3TW, United Kingdom.

出版信息

Proc Natl Acad Sci U S A. 2015 Sep 15;112(37):11708-13. doi: 10.1073/pnas.1505483112. Epub 2015 Aug 31.

Abstract

Research on the dynamics of reward-based, goal-directed decision making has largely focused on simple choice, where participants decide among a set of unitary, mutually exclusive options. Recent work suggests that the deliberation process underlying simple choice can be understood in terms of evidence integration: Noisy evidence in favor of each option accrues over time, until the evidence in favor of one option is significantly greater than the rest. However, real-life decisions often involve not one, but several steps of action, requiring a consideration of cumulative rewards and a sensitivity to recursive decision structure. We present results from two experiments that leveraged techniques previously applied to simple choice to shed light on the deliberation process underlying multistep choice. We interpret the results from these experiments in terms of a new computational model, which extends the evidence accumulation perspective to multiple steps of action.

摘要

基于奖励的目标导向决策动态研究主要集中在简单选择上,即参与者在一组单一、相互排斥的选项中进行决策。最近的研究表明,简单选择背后的审议过程可以通过证据整合来理解:支持每个选项的嘈杂证据会随着时间积累,直到支持一个选项的证据明显大于其他选项。然而,现实生活中的决策通常涉及不止一个行动步骤,需要考虑累积奖励并对递归决策结构保持敏感。我们展示了两项实验的结果,这些实验利用了先前应用于简单选择的技术,以阐明多步选择背后的审议过程。我们根据一个新的计算模型来解释这些实验的结果,该模型将证据积累的观点扩展到多个行动步骤。

相似文献

1
Evidence integration in model-based tree search.
Proc Natl Acad Sci U S A. 2015 Sep 15;112(37):11708-13. doi: 10.1073/pnas.1505483112. Epub 2015 Aug 31.
2
Normative decision rules in changing environments.
Elife. 2022 Oct 25;11:e79824. doi: 10.7554/eLife.79824.
3
Statistical mechanics of reward-modulated learning in decision-making networks.
Neural Comput. 2012 May;24(5):1230-70. doi: 10.1162/NECO_a_00264. Epub 2012 Feb 1.
4
How pupil responses track value-based decision-making during and after reinforcement learning.
PLoS Comput Biol. 2018 Nov 30;14(11):e1006632. doi: 10.1371/journal.pcbi.1006632. eCollection 2018 Nov.
5
How we learn to make decisions: rapid propagation of reinforcement learning prediction errors in humans.
J Cogn Neurosci. 2014 Mar;26(3):635-44. doi: 10.1162/jocn_a_00509. Epub 2013 Oct 29.
6
The drift diffusion model as the choice rule in reinforcement learning.
Psychon Bull Rev. 2017 Aug;24(4):1234-1251. doi: 10.3758/s13423-016-1199-y.
7
Benchmarking for Bayesian Reinforcement Learning.
PLoS One. 2016 Jun 15;11(6):e0157088. doi: 10.1371/journal.pone.0157088. eCollection 2016.
8
Sensorimotor learning biases choice behavior: a learning neural field model for decision making.
PLoS Comput Biol. 2012;8(11):e1002774. doi: 10.1371/journal.pcbi.1002774. Epub 2012 Nov 15.
9
Model-based reinforcement learning under concurrent schedules of reinforcement in rodents.
Learn Mem. 2009 Apr 29;16(5):315-23. doi: 10.1101/lm.1295509. Print 2009 May.
10
Learning the opportunity cost of time in a patch-foraging task.
Cogn Affect Behav Neurosci. 2015 Dec;15(4):837-53. doi: 10.3758/s13415-015-0350-y.

引用本文的文献

1
Disentangling the Component Processes in Complex Planning Impairments Following Ventromedial Prefrontal Lesions.
J Neurosci. 2025 Mar 19;45(12):e1814242025. doi: 10.1523/JNEUROSCI.1814-24.2025.
2
A low-dimensional approximation of optimal confidence.
PLoS Comput Biol. 2024 Jul 24;20(7):e1012273. doi: 10.1371/journal.pcbi.1012273. eCollection 2024 Jul.
3
Transitions in cognitive evolution.
Proc Biol Sci. 2023 Jul 12;290(2002):20230671. doi: 10.1098/rspb.2023.0671. Epub 2023 Jul 5.
5
Neural Mechanisms That Make Perceptual Decisions Flexible.
Annu Rev Physiol. 2023 Feb 10;85:191-215. doi: 10.1146/annurev-physiol-031722-024731. Epub 2022 Nov 7.
6
A weighted constraint satisfaction approach to human goal-directed decision making.
PLoS Comput Biol. 2022 Jun 16;18(6):e1009553. doi: 10.1371/journal.pcbi.1009553. eCollection 2022 Jun.
7
Conflict and competition between model-based and model-free control.
PLoS Comput Biol. 2022 May 5;18(5):e1010047. doi: 10.1371/journal.pcbi.1010047. eCollection 2022 May.
8
Rational use of cognitive resources in human planning.
Nat Hum Behav. 2022 Aug;6(8):1112-1125. doi: 10.1038/s41562-022-01332-8. Epub 2022 Apr 28.
9
Decision prioritization and causal reasoning in decision hierarchies.
PLoS Comput Biol. 2021 Dec 31;17(12):e1009688. doi: 10.1371/journal.pcbi.1009688. eCollection 2021 Dec.
10
Advances in modeling learning and decision-making in neuroscience.
Neuropsychopharmacology. 2022 Jan;47(1):104-118. doi: 10.1038/s41386-021-01126-y. Epub 2021 Aug 27.

本文引用的文献

1
Interplay of approximate planning strategies.
Proc Natl Acad Sci U S A. 2015 Mar 10;112(10):3098-103. doi: 10.1073/pnas.1414219112. Epub 2015 Feb 9.
2
Optimal behavioral hierarchy.
PLoS Comput Biol. 2014 Aug 14;10(8):e1003779. doi: 10.1371/journal.pcbi.1003779. eCollection 2014 Aug.
4
Cortical and hippocampal correlates of deliberation during model-based decisions for rewards in humans.
PLoS Comput Biol. 2013;9(12):e1003387. doi: 10.1371/journal.pcbi.1003387. Epub 2013 Dec 5.
5
Working-memory capacity protects model-based learning from stress.
Proc Natl Acad Sci U S A. 2013 Dec 24;110(52):20941-6. doi: 10.1073/pnas.1312011110. Epub 2013 Dec 9.
6
Disruption of dorsolateral prefrontal cortex decreases model-based in favor of model-free control in humans.
Neuron. 2013 Nov 20;80(4):914-9. doi: 10.1016/j.neuron.2013.08.009. Epub 2013 Oct 24.
7
Goals and habits in the brain.
Neuron. 2013 Oct 16;80(2):312-25. doi: 10.1016/j.neuron.2013.09.007.
8
Simultaneous modeling of visual saliency and value computation improves predictions of economic choice.
Proc Natl Acad Sci U S A. 2013 Oct 1;110(40):E3858-67. doi: 10.1073/pnas.1304429110. Epub 2013 Sep 9.
9
The curse of planning: dissecting multiple reinforcement-learning systems by taxing the central executive.
Psychol Sci. 2013 May;24(5):751-61. doi: 10.1177/0956797612463080. Epub 2013 Apr 4.
10
Hierarchical learning induces two simultaneous, but separable, prediction errors in human basal ganglia.
J Neurosci. 2013 Mar 27;33(13):5797-805. doi: 10.1523/JNEUROSCI.5445-12.2013.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验