• 文献检索
  • 文档翻译
  • 深度研究
  • 学术资讯
  • Suppr Zotero 插件Zotero 插件
  • 邀请有礼
  • 套餐&价格
  • 历史记录
应用&插件
Suppr Zotero 插件Zotero 插件浏览器插件Mac 客户端Windows 客户端微信小程序
定价
高级版会员购买积分包购买API积分包
服务
文献检索文档翻译深度研究API 文档MCP 服务
关于我们
关于 Suppr公司介绍联系我们用户协议隐私条款
关注我们

Suppr 超能文献

核心技术专利:CN118964589B侵权必究
粤ICP备2023148730 号-1Suppr @ 2026

文献检索

告别复杂PubMed语法,用中文像聊天一样搜索,搜遍4000万医学文献。AI智能推荐,让科研检索更轻松。

立即免费搜索

文件翻译

保留排版,准确专业,支持PDF/Word/PPT等文件格式,支持 12+语言互译。

免费翻译文档

深度研究

AI帮你快速写综述,25分钟生成高质量综述,智能提取关键信息,辅助科研写作。

立即免费体验

运动技能习得中的强化学习:利用奖励积极性来理解短期和长期行为适应背后的机制。

Reinforcement learning in motor skill acquisition: using the reward positivity to understand the mechanisms underlying short- and long-term behavior adaptation.

作者信息

Bacelar Mariane F B, Lohse Keith R, Parma Juliana O, Miller Matthew W

机构信息

Department of Kinesiology, Boise State University, Boise, ID, United States.

Program in Physical Therapy, Washington University School of Medicine, St. Louis, MO, United States.

出版信息

Front Behav Neurosci. 2024 Oct 30;18:1466970. doi: 10.3389/fnbeh.2024.1466970. eCollection 2024.

DOI:10.3389/fnbeh.2024.1466970
PMID:39539941
原文链接:https://pmc.ncbi.nlm.nih.gov/articles/PMC11557390/
Abstract

INTRODUCTION

According to reinforcement learning, humans adjust their behavior based on the difference between actual and anticipated outcomes (i.e., prediction error) with the main goal of maximizing rewards through their actions. Despite offering a strong theoretical framework to understand how we acquire motor skills, very few studies have investigated reinforcement learning predictions and its underlying mechanisms in motor skill acquisition.

METHODS

In the present study, we explored a 134-person dataset consisting of learners' feedback-evoked brain activity (reward positivity; RewP) and motor accuracy during the practice phase and delayed retention test to investigate whether these variables interacted according to reinforcement learning predictions.

RESULTS

Results showed a non-linear relationship between RewP and trial accuracy, which was moderated by the learners' performance level. Specifically, high-performing learners were more sensitive to violations in reward expectations compared to low-performing learners, likely because they developed a stronger representation of the skill and were able to rely on more stable outcome predictions. Furthermore, contrary to our prediction, the average RewP during acquisition did not predict performance on the delayed retention test.

DISCUSSION

Together, these findings support the use of reinforcement learning models to understand short-term behavior adaptation and highlight the complexity of the motor skill consolidation process, which would benefit from a multi-mechanistic approach to further our understanding of this phenomenon.

摘要

引言

根据强化学习理论,人类会根据实际结果与预期结果之间的差异(即预测误差)来调整自己的行为,其主要目标是通过行动最大化奖励。尽管强化学习为理解我们如何获得运动技能提供了一个强大的理论框架,但很少有研究探讨强化学习预测及其在运动技能习得中的潜在机制。

方法

在本研究中,我们探索了一个包含134人的数据集,该数据集包括学习者在练习阶段和延迟保留测试期间的反馈诱发脑活动(奖励正性;RewP)和运动准确性,以研究这些变量是否根据强化学习预测相互作用。

结果

结果显示RewP与试验准确性之间存在非线性关系,这种关系受学习者表现水平的调节。具体而言,与低表现学习者相比,高表现学习者对奖励期望的违反更为敏感,这可能是因为他们对技能形成了更强的表征,并且能够依赖更稳定的结果预测。此外,与我们的预测相反,习得过程中的平均RewP并不能预测延迟保留测试中的表现。

讨论

总之,这些发现支持使用强化学习模型来理解短期行为适应,并突出了运动技能巩固过程的复杂性,这将受益于多机制方法,以进一步加深我们对这一现象的理解。

相似文献

1
Reinforcement learning in motor skill acquisition: using the reward positivity to understand the mechanisms underlying short- and long-term behavior adaptation.运动技能习得中的强化学习:利用奖励积极性来理解短期和长期行为适应背后的机制。
Front Behav Neurosci. 2024 Oct 30;18:1466970. doi: 10.3389/fnbeh.2024.1466970. eCollection 2024.
2
Dissociating the contributions of reward-prediction errors to trial-level adaptation and long-term learning.区分奖励预测误差对试验水平适应和长期学习的贡献。
Biol Psychol. 2020 Jan;149:107775. doi: 10.1016/j.biopsycho.2019.107775. Epub 2019 Sep 26.
3
The better, the bigger: The effect of graded positive performance feedback on the reward positivity.越好,越大:分级积极绩效反馈对奖励积极情绪的影响。
Biol Psychol. 2016 Feb;114:61-8. doi: 10.1016/j.biopsycho.2015.12.011. Epub 2016 Jan 3.
4
The reward positivity is sensitive to affective liking.正性奖励敏感于情感喜好。
Cogn Affect Behav Neurosci. 2022 Apr;22(2):258-267. doi: 10.3758/s13415-021-00950-5. Epub 2021 Oct 1.
5
Reinforcement feedback impairs locomotor adaptation and retention.强化反馈会损害运动适应性和记忆保持。
Front Behav Neurosci. 2024 Apr 24;18:1388495. doi: 10.3389/fnbeh.2024.1388495. eCollection 2024.
6
How we learn to make decisions: rapid propagation of reinforcement learning prediction errors in humans.我们如何学习做决策:强化学习预测错误在人类中的快速传播。
J Cogn Neurosci. 2014 Mar;26(3):635-44. doi: 10.1162/jocn_a_00509. Epub 2013 Oct 29.
7
Influence of error-augmentation on the dynamics of visuomotor skill acquisition: insights from proxy-process models.错误增强对运动技能获得动力学的影响:来自代理过程模型的见解。
J Neurophysiol. 2024 Jun 1;131(6):1175-1187. doi: 10.1152/jn.00051.2024. Epub 2024 May 1.
8
The aversion positivity: Mediofrontal cortical potentials reflect parametric aversive prediction errors and drive behavioral modification following negative reinforcement.厌恶正性化:额眶部皮质电势反映了参数性厌恶预测误差,并在负强化后驱动行为修正。
Cortex. 2021 Jul;140:26-39. doi: 10.1016/j.cortex.2021.03.012. Epub 2021 Mar 27.
9
Prediction-error-dependent processing of immediate and delayed positive feedback.即时和延迟正反馈的预测误差依赖性处理。
Sci Rep. 2024 Apr 27;14(1):9674. doi: 10.1038/s41598-024-60328-8.
10
Pain feedback interferes with reward positivity production.疼痛反馈会干扰奖励正性波的产生。
Psychophysiology. 2022 Jun;59(6):e14004. doi: 10.1111/psyp.14004. Epub 2022 Feb 19.

本文引用的文献

1
Fundamental processes in sensorimotor learning: Reasoning, refinement, and retrieval.感觉运动学习的基本过程:推理、优化与检索。
Elife. 2024 Aug 1;13:e91839. doi: 10.7554/eLife.91839.
2
Relationship between reward-related brain activity and opportunities to sit.与奖励相关的大脑活动与久坐机会之间的关系。
Cortex. 2023 Oct;167:197-217. doi: 10.1016/j.cortex.2023.06.011. Epub 2023 Jul 24.
3
Error-based and reinforcement learning in basketball free throw shooting.基于错误的和强化学习在篮球罚球中的应用。
Sci Rep. 2023 Jan 10;13(1):499. doi: 10.1038/s41598-022-26568-2.
4
Archery under the (electroencephalography-)hood: Theta-lateralization as a marker for motor learning.头戴式脑电帽下的射箭:θ侧化作为运动学习的标志物。
Neuroscience. 2022 Sep 1;499:23-39. doi: 10.1016/j.neuroscience.2022.07.019. Epub 2022 Jul 21.
5
An extended challenge-based framework for practice design in sports coaching.一个用于体育教练实践设计的基于挑战的扩展框架。
J Sports Sci. 2022 Apr;40(7):754-768. doi: 10.1080/02640414.2021.2015917. Epub 2022 Jan 12.
6
Long-term motor skill training with individually adjusted progressive difficulty enhances learning and promotes corticospinal plasticity.长期进行个体调整递增难度的运动技能训练可以增强学习能力并促进皮质脊髓可塑性。
Sci Rep. 2020 Sep 24;10(1):15588. doi: 10.1038/s41598-020-72139-8.
7
Dissociating the contributions of reward-prediction errors to trial-level adaptation and long-term learning.区分奖励预测误差对试验水平适应和长期学习的贡献。
Biol Psychol. 2020 Jan;149:107775. doi: 10.1016/j.biopsycho.2019.107775. Epub 2019 Sep 26.
8
Internal Models in Biological Control.生物控制中的内部模型
Annu Rev Control Robot Auton Syst. 2019 May 1;2:339-364. doi: 10.1146/annurev-control-060117-105206.
9
Neural signatures of reward and sensory error feedback processing in motor learning.运动学习中奖励和感觉错误反馈处理的神经特征。
J Neurophysiol. 2019 Apr 1;121(4):1561-1574. doi: 10.1152/jn.00792.2018. Epub 2019 Feb 27.
10
Contribution of explicit processes to reinforcement-based motor learning.显性过程对基于强化的运动学习的贡献。
J Neurophysiol. 2018 Jun 1;119(6):2241-2255. doi: 10.1152/jn.00901.2017. Epub 2018 Mar 14.