Chien Samson, Wiehler Antonius, Spezio Michael, Gläscher Jan
Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg 20246, Germany, and
Institute for Systems Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg 20246, Germany, and.
J Neurosci. 2016 May 4;36(18):5003-12. doi: 10.1523/JNEUROSCI.3084-15.2016.
Most real-life cues exhibit certain inherent values that may interfere with or facilitate the acquisition of new expected values during associative learning. In particular, when inherent and acquired values are congruent, learning may progress more rapidly. Here we investigated such an influence through a 2 × 2 factorial design, using attractiveness (high/low) of the facial picture as a proxy for the inherent value of the cue and its reward probability (high/low) as a surrogate for the acquired value. Each picture was paired with a monetary win or loss either congruently or incongruently. Behavioral results from 32 human participants indicated both faster response time and faster learning rate for value-congruent cue-outcome pairings. Model-based fMRI analysis revealed a fractionation of reinforcement learning (RL) signals in the ventral striatum, including a strong and novel correlation between the cue-specific decaying learning rate and BOLD activity in the ventral caudate. Additionally, we detected a functional link between neural signals of both learning rate and reward prediction error in the ventral striatum, and the signal of expected value in the ventromedial prefrontal cortex, showing a novel confirmation of the mathematical RL model via functional connectivity.
Most real-world decisions require the integration of inherent value and sensitivity to outcomes to facilitate adaptive learning. Inherent value is drawing increasing interest from decision scientists because it influences decisions in contexts ranging from advertising to investing. This study provides novel insight into how inherent value influences the acquisition of new expected value during associative learning. Specifically, we find that the congruence between the inherent value and the acquired reward influences the neural coding of learning rate. We also show for the first time that neuroimaging signals coding the learning rate, prediction error, and acquired value follow the multiplicative Rescorla-Wagner learning rule, a finding predicted by reinforcement learning theory.
大多数现实生活中的线索都具有某些内在价值，这些价值可能会在联想学习过程中干扰或促进新预期价值的获取。特别是，当内在价值和习得价值一致时，学习可能会进展得更快。在这里，我们通过2×2析因设计研究了这种影响，使用面部图片的吸引力（高/低）作为线索内在价值的代理，其奖励概率（高/低）作为习得价值的替代。每张图片都与金钱输赢进行一致或不一致的配对。32名人类参与者的行为结果表明，价值一致的线索-结果配对的反应时间更快，学习速度也更快。基于模型的功能磁共振成像分析揭示了腹侧纹状体中强化学习（RL）信号的分离，包括线索特异性衰减学习率与腹侧尾状核中的BOLD活动之间存在强烈且新颖的相关性。此外，我们检测到腹侧纹状体中学习率和奖励预测误差的神经信号与腹内侧前额叶皮层中预期价值信号之间的功能联系，通过功能连接展示了对数学RL模型的新证实。
大多数现实世界的决策需要整合内在价值和对结果的敏感性，以促进适应性学习。内在价值正越来越受到决策科学家的关注，因为它在从广告到投资等各种情境中影响决策。这项研究为内在价值如何在联想学习过程中影响新预期价值的获取提供了新的见解。具体而言，我们发现内在价值与习得奖励之间的一致性会影响学习率的神经编码。我们还首次表明，编码学习率、预测误差和习得价值的神经成像信号遵循乘法雷斯克拉-瓦格纳学习规则，这是强化学习理论预测的一个发现。
