Suppr超能文献

主观和模型估计的奖励预测:与强化学习任务中的反馈相关负波 (FRN) 和奖励预测误差的关联。

Subjective and model-estimated reward prediction: association with the feedback-related negativity (FRN) and reward prediction error in a reinforcement learning task.

机构信息

Department of Psychology, Nagoya University, Japan.

出版信息

Int J Psychophysiol. 2010 Dec;78(3):273-83. doi: 10.1016/j.ijpsycho.2010.09.001. Epub 2010 Sep 19.

Abstract

In this study, we examined whether the feedback-related negativity (FRN) is associated with both subjective and objective (model-estimated) reward prediction errors (RPE) per trial in a reinforcement learning task in healthy adults (n=25). The level of RPE was assessed by 1) subjective ratings per trial and by 2) a computational model of reinforcement learning. As results, model-estimated RPE was highly correlated with subjective RPE (r=.82), and the grand-averaged ERP waves based on the trials with high and low model-estimated RPE showed the significant difference only in the time period of the FRN component (p<.05). Regardless of the time course of learning, FRN was associated with both subjective and model-estimated RPEs within subject (r=.47, p<.001; r=.40, p<.05) and between subjects (r=.33, p<.05; r=.41, p<.005) only in the Learnable condition where the internal reward prediction varied enough with a behavior-reward contingency.

摘要

在这项研究中,我们在一项强化学习任务中,检查了健康成年人(n=25)中每一次试验的反馈相关负波(FRN)是否与主观和客观(模型估计)奖励预测误差(RPE)都有关联。RPE 的水平通过以下方式评估:1)每次试验的主观评分和 2)强化学习的计算模型。结果表明,模型估计的 RPE 与主观 RPE 高度相关(r=.82),并且基于模型估计的 RPE 高低的试验的平均 ERP 波仅在 FRN 成分的时间段内显示出显著差异(p<.05)。无论学习的时间进程如何,FRN 仅在可学习条件下,与主观和模型估计的 RPE 之间存在关联,在可学习条件下,内部奖励预测与行为奖励关联的变化足以引起 FRN(个体内 r=.47,p<.001;r=.40,p<.05;个体间 r=.33,p<.05;r=.41,p<.005)。

相似文献

2
Valence-separated representation of reward prediction error in feedback-related negativity and positivity.
Neuroreport. 2015 Feb 11;26(3):157-62. doi: 10.1097/WNR.0000000000000318.
3
When the outcome is different than expected: Subjective expectancy shapes reward prediction error at the FRN level.
Psychophysiology. 2019 Dec;56(12):e13456. doi: 10.1111/psyp.13456. Epub 2019 Aug 12.
4
Principal components analysis of reward prediction errors in a reinforcement learning task.
Neuroimage. 2016 Jan 1;124(Pt A):276-286. doi: 10.1016/j.neuroimage.2015.07.032. Epub 2015 Jul 18.
5
Better or worse than expected? Aging, learning, and the ERN.
Neuropsychologia. 2008 Jan 31;46(2):521-39. doi: 10.1016/j.neuropsychologia.2007.09.001. Epub 2007 Sep 7.
6
The Feedback-related Negativity Codes Components of Abstract Inference during Reward-based Decision-making.
J Cogn Neurosci. 2016 Aug;28(8):1127-38. doi: 10.1162/jocn_a_00957. Epub 2016 Mar 31.
7
Mediofrontal event-related potentials in response to positive, negative and unsigned prediction errors.
Neuropsychologia. 2014 Aug;61:1-10. doi: 10.1016/j.neuropsychologia.2014.06.004. Epub 2014 Jun 16.
8
EEG correlates of physical effort and reward processing during reinforcement learning.
J Neurophysiol. 2020 Aug 1;124(2):610-622. doi: 10.1152/jn.00370.2020. Epub 2020 Jul 29.
9
Probabilistic reward learning in adults with Attention Deficit Hyperactivity Disorder--an electrophysiological study.
Psychiatry Res. 2015 Jan 30;225(1-2):133-144. doi: 10.1016/j.psychres.2014.11.006. Epub 2014 Nov 13.
10
Aberrant reward prediction errors in young adult at-risk alcohol users.
Addict Biol. 2021 Jan;26(1):e12873. doi: 10.1111/adb.12873. Epub 2020 Jan 23.

引用本文的文献

3
The cerebellum contributes to prediction error coding in reinforcement learning in humans.
J Neurosci. 2025 Mar 26;45(19). doi: 10.1523/JNEUROSCI.1972-24.2025.
5
Revisiting the importance of model fitting for model-based fMRI: It does matter in computational psychiatry.
PLoS Comput Biol. 2021 Feb 9;17(2):e1008738. doi: 10.1371/journal.pcbi.1008738. eCollection 2021 Feb.
6
Visual Feedback Modulates Aftereffects and Electrophysiological Markers of Prism Adaptation.
Front Hum Neurosci. 2020 Apr 17;14:138. doi: 10.3389/fnhum.2020.00138. eCollection 2020.
8
Frontal midline theta reflects anxiety and cognitive control: meta-analytic evidence.
J Physiol Paris. 2015 Feb-Jun;109(1-3):3-15. doi: 10.1016/j.jphysparis.2014.04.003. Epub 2014 Apr 29.
9
Learning from experience: event-related potential correlates of reward processing, neural adaptation, and behavioral choice.
Neurosci Biobehav Rev. 2012 Sep;36(8):1870-84. doi: 10.1016/j.neubiorev.2012.05.008. Epub 2012 Jun 7.
10
Frontal theta reflects uncertainty and unexpectedness during exploration and exploitation.
Cereb Cortex. 2012 Nov;22(11):2575-86. doi: 10.1093/cercor/bhr332. Epub 2011 Nov 25.

本文引用的文献

1
Brain and autonomic association accompanying stochastic decision-making.
Neuroimage. 2010 Jan 1;49(1):1024-37. doi: 10.1016/j.neuroimage.2009.07.060. Epub 2009 Jul 30.
2
The stability of error-related brain activity with increasing trials.
Psychophysiology. 2009 Sep;46(5):957-61. doi: 10.1111/j.1469-8986.2009.00848.x. Epub 2009 Jun 23.
3
When is an error not a prediction error? An electrophysiological investigation.
Cogn Affect Behav Neurosci. 2009 Mar;9(1):59-70. doi: 10.3758/CABN.9.1.59.
4
The neural consequences of flip-flopping: the feedback-related negativity and salience of reward prediction.
Psychophysiology. 2009 Mar;46(2):313-20. doi: 10.1111/j.1469-8986.2008.00760.x. Epub 2009 Jan 20.
5
Learning to become an expert: reinforcement learning and the acquisition of perceptual expertise.
J Cogn Neurosci. 2009 Sep;21(9):1834-41. doi: 10.1162/jocn.2009.21128.
6
Blink before and after you think: blinks occur prior to and following cognitive load indexed by pupillary responses.
Psychophysiology. 2008 Sep;45(5):679-87. doi: 10.1111/j.1469-8986.2008.00681.x. Epub 2008 Jul 24.
7
Individual differences in reinforcement learning: behavioral, electrophysiological, and neuroimaging correlates.
Neuroimage. 2008 Aug 15;42(2):807-16. doi: 10.1016/j.neuroimage.2008.05.032. Epub 2008 Jul 2.
8
The feedback correct-related positivity: sensitivity of the event-related brain potential to unexpected positive feedback.
Psychophysiology. 2008 Sep;45(5):688-97. doi: 10.1111/j.1469-8986.2008.00668.x. Epub 2008 May 30.
9
Better or worse than expected? Aging, learning, and the ERN.
Neuropsychologia. 2008 Jan 31;46(2):521-39. doi: 10.1016/j.neuropsychologia.2007.09.001. Epub 2007 Sep 7.
10
Optimization by simulated annealing.
Science. 1983 May 13;220(4598):671-80. doi: 10.1126/science.220.4598.671.

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验