Department of Psychology, University of South Carolina.
J Exp Psychol Learn Mem Cogn. 2023 Aug;49(8):1193-1217. doi: 10.1037/xlm0001145. Epub 2022 Jul 4.
In reinforcement learning (RL) tasks, decision makers learn the values of actions in a context-dependent fashion. Although context dependence has many advantages, it can lead to suboptimal preferences when choice options are extrapolated beyond their original encoding contexts. Here, we tested whether we could manipulate context dependence in RL by introducing a secondary task designed to bias attention toward either absolute or relative outcomes. Participants completed a learning phase that involved choices between two (Experiment 1; = 111) or three (Experiment 2; = 90) options per trial with complete feedback. Choice options were grouped in stable contexts so that only a small set of the possible combinations were encountered. One group of participants rated how they felt about particular options (Feelings condition), and another group reported how much they expected to win from particular options (Outcomes condition) at occasional points throughout the learning phase. A third group (Control condition) made no ratings. In the subsequent transfer test, participants chose between all possible pairs of options without feedback. The experimental manipulation had no effect on learning phase performance but a significant effect on transfer, with the Feelings and Control conditions exhibiting greater context dependence than the Outcomes condition. Further, rated feelings reflected relative valuation whereas expected outcomes were more sensitive to absolute option values. Hierarchical Bayesian modeling was used to summarize the findings from both experiments. Our results suggest that attending to affective reactions versus expected outcomes moderates the effects of encoding context on subsequent choices. (PsycInfo Database Record (c) 2023 APA, all rights reserved).
在强化学习（RL）任务中，决策者以依赖上下文的方式学习动作的价值。尽管上下文依赖性有许多优点，但当选择选项超出其原始编码上下文进行推断时，它可能导致次优偏好。在这里，我们通过引入一项旨在偏向绝对或相对结果的辅助任务来测试我们是否可以在 RL 中操纵上下文依赖性。参与者完成了一个学习阶段，该阶段涉及每次试验在两个（实验 1；n = 111）或三个（实验 2；n = 90）选项之间进行选择，并提供完整反馈。选择选项按稳定的上下文分组，因此只遇到一小部分可能的组合。一组参与者对特定选项的感觉进行了评分（感觉条件），另一组参与者在学习阶段的偶尔点报告他们期望从特定选项中赢得多少（结果条件）。第三组（对照组）没有评分。在随后的转移测试中，参与者在没有反馈的情况下选择所有可能的选项对。实验操作对学习阶段的表现没有影响，但对转移有显著影响，感觉条件和对照组比结果条件表现出更大的上下文依赖性。此外，评分的感觉反映了相对估值，而预期的结果对绝对选项值更为敏感。分层贝叶斯模型用于总结两个实验的结果。我们的研究结果表明，关注情感反应与预期结果会调节编码上下文对后续选择的影响。（PsycInfo 数据库记录（c）2023 APA，保留所有权利）。
