• 文献检索
  • 文档翻译
  • 深度研究
  • 学术资讯
  • Suppr Zotero 插件Zotero 插件
  • 邀请有礼
  • 套餐&价格
  • 历史记录
应用&插件
Suppr Zotero 插件Zotero 插件浏览器插件Mac 客户端Windows 客户端微信小程序
定价
高级版会员购买积分包购买API积分包
服务
文献检索文档翻译深度研究API 文档MCP 服务
关于我们
关于 Suppr公司介绍联系我们用户协议隐私条款
关注我们

Suppr 超能文献

核心技术专利:CN118964589B侵权必究
粤ICP备2023148730 号-1Suppr @ 2026

文献检索

告别复杂PubMed语法,用中文像聊天一样搜索,搜遍4000万医学文献。AI智能推荐,让科研检索更轻松。

立即免费搜索

文件翻译

保留排版,准确专业,支持PDF/Word/PPT等文件格式,支持 12+语言互译。

免费翻译文档

深度研究

AI帮你快速写综述,25分钟生成高质量综述,智能提取关键信息,辅助科研写作。

立即免费体验

潜在未选动作价值更新的计算机制。

Computational mechanisms underlying latent value updating of unchosen actions.

机构信息

School of Psychological Sciences, Tel Aviv University, Tel Aviv, Israel.

Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel.

出版信息

Sci Adv. 2023 Oct 20;9(42):eadi2704. doi: 10.1126/sciadv.adi2704.

DOI:10.1126/sciadv.adi2704
PMID:37862419
原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC10588947/
Abstract

Current studies suggest that individuals estimate the value of their choices based on observed feedback. Here, we ask whether individuals also update the value of their unchosen actions, even when the associated feedback remains unknown. One hundred seventy-eight individuals completed a multi-armed bandit task, making choices to gain rewards. We found robust evidence suggesting latent value updating of unchosen actions based on the chosen action's outcome. Computational modeling results suggested that this effect is mainly explained by a value updating mechanism whereby individuals integrate the outcome history for choosing an option with that of rejecting the alternative. Properties of the deliberation (i.e., duration/difficulty) did not moderate the latent value updating of unchosen actions, suggesting that memory traces generated during deliberation might take a smaller role in this specific phenomenon than previously thought. We discuss the mechanisms facilitating credit assignment to unchosen actions and their implications for human decision-making.

摘要

目前的研究表明,个体基于观察到的反馈来估计其选择的价值。在这里,我们想知道个体是否也会更新未选中的动作的价值,即使相关反馈仍然未知。178 名个体完成了一项多臂赌博任务,通过选择来获得奖励。我们发现了强有力的证据表明,基于所选动作的结果,对未选中的动作进行潜在的价值更新。计算建模结果表明,这种效应主要是由一种价值更新机制解释的,即个体将选择一个选项的结果历史与拒绝另一个选项的结果历史进行整合。审议的特性(即持续时间/难度)并没有调节未选中的动作的潜在价值更新,这表明在这个特定现象中,审议期间生成的记忆痕迹可能比之前认为的作用更小。我们讨论了促进对未选中的动作进行信用分配的机制及其对人类决策的影响。

https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/b36e60c274d3/sciadv.adi2704-f5.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/89c944352774/sciadv.adi2704-f1.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/a7cd227b2e58/sciadv.adi2704-f2.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/f066878c88d5/sciadv.adi2704-f3.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/a87fba700e3d/sciadv.adi2704-f4.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/b36e60c274d3/sciadv.adi2704-f5.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/89c944352774/sciadv.adi2704-f1.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/a7cd227b2e58/sciadv.adi2704-f2.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/f066878c88d5/sciadv.adi2704-f3.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/a87fba700e3d/sciadv.adi2704-f4.jpg
https://cdn.ncbi.nlm.nih.gov/pmc/blobs/122b/10588947/b36e60c274d3/sciadv.adi2704-f5.jpg

相似文献

1
Computational mechanisms underlying latent value updating of unchosen actions.潜在未选动作价值更新的计算机制。
Sci Adv. 2023 Oct 20;9(42):eadi2704. doi: 10.1126/sciadv.adi2704.
2
A Normative Account of Confirmation Bias During Reinforcement Learning.强化学习中确认偏差的规范解释。
Neural Comput. 2022 Jan 14;34(2):307-337. doi: 10.1162/neco_a_01455.
3
Devaluation of Unchosen Options: A Bayesian Account of the Provenance and Maintenance of Overly Optimistic Expectations.未被选择选项的贬值:对过度乐观期望的起源与维持的贝叶斯解释。
Cogsci. 2020 Jul-Aug;42:1682-1688.
4
Abstract inference of unchosen option values.未选择选项值的抽象推理。
Eur J Neurosci. 2020 Feb;51(3):909-921. doi: 10.1111/ejn.14577. Epub 2019 Oct 23.
5
The Feedback-related Negativity Codes Components of Abstract Inference during Reward-based Decision-making.基于奖励的决策过程中,反馈相关负波编码抽象推理的组成部分。
J Cogn Neurosci. 2016 Aug;28(8):1127-38. doi: 10.1162/jocn_a_00957. Epub 2016 Mar 31.
6
Implicit Counterfactual Effect in Partial Feedback Reinforcement Learning: Behavioral and Modeling Approach.部分反馈强化学习中的内隐反事实效应:行为与建模方法
Front Neurosci. 2022 May 10;16:631347. doi: 10.3389/fnins.2022.631347. eCollection 2022.
7
The Computational Development of Reinforcement Learning during Adolescence.青少年时期强化学习的计算发展
PLoS Comput Biol. 2016 Jun 20;12(6):e1004953. doi: 10.1371/journal.pcbi.1004953. eCollection 2016 Jun.
8
Improving memory through choice and deliberation in decision-making: Evidence from ERPs.通过决策中的选择和深思来改善记忆:来自事件相关电位的证据。
Psychophysiology. 2025 Jan;62(1):e14662. doi: 10.1111/psyp.14662. Epub 2024 Jul 30.
9
Memory and decision making interact to shape the value of unchosen options.记忆和决策相互作用,共同塑造未被选择选项的价值。
Nat Commun. 2021 Jul 30;12(1):4648. doi: 10.1038/s41467-021-24907-x.
10
The role of memory in counterfactual valuation.记忆在反事实估值中的作用。
J Exp Psychol Gen. 2023 Jun;152(6):1754-1767. doi: 10.1037/xge0001364. Epub 2023 May 18.

引用本文的文献

1
How working memory and reinforcement learning interact when avoiding punishment and pursuing reward concurrently.当同时避免惩罚和追求奖励时,工作记忆与强化学习是如何相互作用的。
J Exp Psychol Gen. 2025 Sep 1. doi: 10.1037/xge0001817.
2
The more random people's preference judgments are, the more they explore in gambling tasks.人们的偏好判断越随机,他们在赌博任务中探索得就越多。
BMC Psychol. 2024 Dec 20;12(1):766. doi: 10.1186/s40359-024-02252-0.
3
Cognitive psychology: computing the value of the choices we do not make.认知心理学:计算我们未做出的选择的价值。

本文引用的文献

1
A neural mechanism for terminating decisions.终止决策的神经机制。
Neuron. 2023 Aug 16;111(16):2601-2613.e5. doi: 10.1016/j.neuron.2023.05.028. Epub 2023 Jun 22.
2
The role of memory in counterfactual valuation.记忆在反事实估值中的作用。
J Exp Psychol Gen. 2023 Jun;152(6):1754-1767. doi: 10.1037/xge0001364. Epub 2023 May 18.
3
Error monitoring when no errors are possible: Arbitrary free-choice decisions invoke error monitoring processes.在不可能出现错误时进行错误监测:任意的自由选择决策会引发错误监测过程。
Commun Psychol. 2024 Mar 2;2(1):17. doi: 10.1038/s44271-024-00064-x.
4
From Tripping and Falling to Ruminating and Worrying: A Meta-Control Account of Repetitive Negative Thinking.从绊倒跌倒到反复思考与担忧:重复性消极思维的元控制理论
Curr Opin Behav Sci. 2024 Apr;56. doi: 10.1016/j.cobeha.2024.101356. Epub 2024 Feb 16.
5
Flexible gating between subspaces in a neural network model of internally guided task switching.内部引导任务切换神经网络模型中子空间之间的灵活门控。
Nat Commun. 2024 Aug 1;15(1):6497. doi: 10.1038/s41467-024-50501-y.
6
Value Estimation versus Effort Mobilization: A General Dissociation between Ventromedial and Dorsomedial Prefrontal Cortex.价值估计与努力调动:腹内侧前额叶皮层和背内侧前额叶皮层之间的一般分离
J Neurosci. 2024 Apr 24;44(17):e1176232024. doi: 10.1523/JNEUROSCI.1176-23.2024.
7
Flexible gating between subspaces in a neural network model of internally guided task switching.内部引导任务切换神经网络模型中各子空间之间的灵活门控。
bioRxiv. 2024 Jun 10:2023.08.15.553375. doi: 10.1101/2023.08.15.553375.
iScience. 2023 Mar 10;26(4):106373. doi: 10.1016/j.isci.2023.106373. eCollection 2023 Apr 21.
4
Formation of non-veridical action-outcome associations following exposure to threat-related cues.接触与威胁相关的线索后形成非真实的动作-结果关联。
Emotion. 2023 Oct;23(7):2094-2099. doi: 10.1037/emo0001212. Epub 2023 Mar 13.
5
Agency as a bridge to form associative memories.作为形成联想记忆的桥梁的机构。
J Exp Psychol Gen. 2023 Jun;152(6):1797-1813. doi: 10.1037/xge0001356. Epub 2023 Mar 2.
6
Stan: A Probabilistic Programming Language.斯坦:一种概率编程语言。
J Stat Softw. 2017;76. doi: 10.18637/jss.v076.i01. Epub 2017 Jan 11.
7
The computational roots of positivity and confirmation biases in reinforcement learning.强化学习中积极性和确认偏见的计算根源。
Trends Cogn Sci. 2022 Jul;26(7):607-621. doi: 10.1016/j.tics.2022.04.005. Epub 2022 May 31.
8
Why unchosen options linger in our minds.为什么未被选中的选项会在我们的脑海中挥之不去。
Commun Biol. 2021 Nov 8;4(1):1271. doi: 10.1038/s42003-021-02803-w.
9
Mixing memory and desire: How memory reactivation supports deliberative decision-making.混合记忆与欲望:记忆再激活如何支持审慎决策。
Wiley Interdiscip Rev Cogn Sci. 2022 Mar;13(2):e1581. doi: 10.1002/wcs.1581. Epub 2021 Oct 19.
10
The elusive effects of incidental anxiety on reinforcement-learning.偶发性焦虑对强化学习的难以捉摸的影响。
J Exp Psychol Learn Mem Cogn. 2022 May;48(5):619-642. doi: 10.1037/xlm0001033. Epub 2021 Sep 13.