• 文献检索
  • 文档翻译
  • 深度研究
  • 学术资讯
  • Suppr Zotero 插件Zotero 插件
  • 邀请有礼
  • 套餐&价格
  • 历史记录
应用&插件
Suppr Zotero 插件Zotero 插件浏览器插件Mac 客户端Windows 客户端微信小程序
定价
高级版会员购买积分包购买API积分包
服务
文献检索文档翻译深度研究API 文档MCP 服务
关于我们
关于 Suppr公司介绍联系我们用户协议隐私条款
关注我们

Suppr 超能文献

核心技术专利:CN118964589B侵权必究
粤ICP备2023148730 号-1Suppr @ 2026

文献检索

告别复杂PubMed语法,用中文像聊天一样搜索,搜遍4000万医学文献。AI智能推荐,让科研检索更轻松。

立即免费搜索

文件翻译

保留排版,准确专业,支持PDF/Word/PPT等文件格式,支持 12+语言互译。

免费翻译文档

深度研究

AI帮你快速写综述,25分钟生成高质量综述,智能提取关键信息,辅助科研写作。

立即免费体验

新颖性对强化学习的影响。

The effect of novelty on reinforcement learning.

机构信息

Bernstein Center for Computational Neuroscience, Philippstr, Haus, Berlin, Germany.

出版信息

Prog Brain Res. 2013;202:415-39. doi: 10.1016/B978-0-444-62604-2.00021-6.

DOI:10.1016/B978-0-444-62604-2.00021-6
PMID:23317843
Abstract

Recent research suggests that novelty has an influence on reward-related learning. Here, we showed that novel stimuli presented from a pre-familiarized category can accelerate or decelerate learning of the most rewarding category, depending on the condition. The extent of this influence depended on the individual trait of novelty seeking. Different reinforcement learning models were developed to quantify subjects' choices. We introduced a bias parameter to model explorative behavior toward novel stimuli and characterize individual variation in novelty response. The theoretical framework allowed us to test different assumptions, concerning the motivational value of novelty. The best fitting-model combined all novelty components and had a significant positive correlation with both the experimentally measured novelty bias and the independent novelty-seeking trait. Altogether, we have not only shown that novelty by itself enhances behavioral responses underlying reward processing, but also that novelty has a direct influence on reward-dependent learning processes, consistently with computational predictions.

摘要

最近的研究表明,新奇性对与奖励相关的学习有影响。在这里,我们表明,根据条件的不同,从预先熟悉的类别中呈现的新刺激可以加速或减缓最有奖励性的类别学习。这种影响的程度取决于个体对新奇的寻求特质。开发了不同的强化学习模型来量化被试的选择。我们引入了一个偏差参数来对新刺激的探索性行为进行建模,并对个体对新奇的反应进行特征化。该理论框架允许我们测试关于新奇性的动机价值的不同假设。拟合最好的模型结合了所有新奇性成分,与实验测量的新奇性偏差和独立的新奇性寻求特征都有显著的正相关。总的来说,我们不仅表明新奇性本身可以增强奖励处理的基础行为反应,而且新奇性对依赖奖励的学习过程有直接影响,这与计算预测一致。

相似文献

1
The effect of novelty on reinforcement learning.新颖性对强化学习的影响。
Prog Brain Res. 2013;202:415-39. doi: 10.1016/B978-0-444-62604-2.00021-6.
2
The novelty exploration bonus and its attentional modulation.新奇探索奖励及其注意力调节。
Neuropsychologia. 2009 Sep;47(11):2272-81. doi: 10.1016/j.neuropsychologia.2009.01.015. Epub 2009 Jan 19.
3
Novelty is not surprise: Human exploratory and adaptive behavior in sequential decision-making.新颖性不是惊喜:人类在序列决策中的探索和适应行为。
PLoS Comput Biol. 2021 Jun 3;17(6):e1009070. doi: 10.1371/journal.pcbi.1009070. eCollection 2021 Jun.
4
Reinforcement learning and decision making in monkeys during a competitive game.猴子在竞争性游戏中的强化学习与决策
Brain Res Cogn Brain Res. 2004 Dec;22(1):45-58. doi: 10.1016/j.cogbrainres.2004.07.007.
5
The actor-critic learning is behind the matching law: matching versus optimal behaviors.行动者-评论家学习是匹配法则背后的原理:匹配行为与最优行为。
Neural Comput. 2008 Jan;20(1):227-51. doi: 10.1162/neco.2008.20.1.227.
6
Model-based reinforcement learning under concurrent schedules of reinforcement in rodents.啮齿动物在并发强化程序下基于模型的强化学习
Learn Mem. 2009 Apr 29;16(5):315-23. doi: 10.1101/lm.1295509. Print 2009 May.
7
Adaptive learning via selectionism and Bayesianism, Part II: the sequential case.基于选择主义和贝叶斯主义的适应性学习,第二部分:序列情形。
Neural Netw. 2009 Apr;22(3):229-36. doi: 10.1016/j.neunet.2009.03.017. Epub 2009 Apr 5.
8
Attentional bias as trait: correlations with novelty seeking.作为特质的注意偏向:与寻求新奇性的相关性。
Neuropsychologia. 2008;46(7):2064-70. doi: 10.1016/j.neuropsychologia.2008.02.005. Epub 2008 Feb 9.
9
Hierarchical reinforcement learning and decision making.分层强化学习与决策。
Curr Opin Neurobiol. 2012 Dec;22(6):956-62. doi: 10.1016/j.conb.2012.05.008. Epub 2012 Jun 11.
10
Dopamine Regulates Approach-Avoidance in Human Sensation-Seeking.多巴胺调节人类寻求刺激行为中的趋近-回避行为。
Int J Neuropsychopharmacol. 2015 Apr 9;18(10):pyv041. doi: 10.1093/ijnp/pyv041.

引用本文的文献

1
Perceptual Novelty Drives Early Exploration in a Bottom-Up Manner.感知新奇性以自下而上的方式驱动早期探索。
Dev Sci. 2025 May;28(3):e70002. doi: 10.1111/desc.70002.
2
The neighborhood walk: introducing first-year medical students to social determinants of health in underserved neighborhoods.社区漫步:向一年级医学生介绍服务欠缺社区的健康社会决定因素。
BMC Med Educ. 2025 Jan 31;25(1):163. doi: 10.1186/s12909-025-06743-5.
3
Feasibility of At-Home Hand Arm Bimanual Intensive Training in Virtual Reality: Case Study.家庭式双手手臂虚拟现实强化训练的可行性:案例研究。
JMIR Form Res. 2024 Sep 6;8:e57588. doi: 10.2196/57588.
4
Curiosity: primate neural circuits for novelty and information seeking.好奇心:灵长类动物用于寻求新奇和信息的神经回路。
Nat Rev Neurosci. 2024 Mar;25(3):195-208. doi: 10.1038/s41583-023-00784-9. Epub 2024 Jan 23.
5
Harnessing technology and gamification to increase adult physical activity: a cluster randomized controlled trial of the Columbia Moves pilot.利用技术和游戏化提高成年人的身体活动水平:哥伦比亚行动试点的一项集群随机对照试验。
Int J Behav Nutr Phys Act. 2023 Nov 3;20(1):129. doi: 10.1186/s12966-023-01530-1.
6
Novelty and uncertainty differentially drive exploration across development.新颖性和不确定性在发展过程中以不同的方式驱动探索。
Elife. 2023 Aug 16;12:e84260. doi: 10.7554/eLife.84260.
7
Systemic kappa opioid receptor antagonism accelerates reinforcement learning via augmentation of novelty processing in male mice.系统 κ 阿片受体拮抗作用通过增强雄性小鼠新奇处理来加速强化学习。
Neuropsychopharmacology. 2023 May;48(6):857-868. doi: 10.1038/s41386-023-01547-x. Epub 2023 Feb 17.
8
Individual differences in naturalistic learning link negative emotionality to the development of anxiety.个体在自然学习中的差异将负性情绪与焦虑的发展联系起来。
Sci Adv. 2023 Jan 4;9(1):eadd2976. doi: 10.1126/sciadv.add2976.
9
Rats use strategies to make object choices in spontaneous object recognition tasks.大鼠在自发物体识别任务中使用策略来进行物体选择。
Sci Rep. 2022 Oct 10;12(1):16973. doi: 10.1038/s41598-022-21537-1.
10
Virtual Reality vs. Tablet Video as an Experiential Education Platform for Pediatric Patients Undergoing Chest Radiography: A Randomized Clinical Trial.虚拟现实与平板电脑视频作为接受胸部X光检查的儿科患者的体验式教育平台:一项随机临床试验
J Clin Med. 2021 Jun 4;10(11):2486. doi: 10.3390/jcm10112486.