Chase Juliana, Xia Liyu, Tai Lung-Hao, Lin Wan Chen, Collins Anne G E, Wilbrecht Linda
Department of Psychology, University of California, Berkeley, Berkeley, California, United States of America.
Department of Neuroscience, University of California, Berkeley, Berkeley, California, United States of America.
PLoS Comput Biol. 2024 Dec 23;20(12):e1012667. doi: 10.1371/journal.pcbi.1012667. eCollection 2024 Dec.
Computational modeling has revealed that human research participants use both rapid working memory (WM) and incremental reinforcement learning (RL) (RL+WM) to solve a simple instrumental learning task, relying on WM when the number of stimuli is small and supplementing with RL when the number of stimuli exceeds WM capacity. Inspired by this work, we examined which learning systems and strategies are used by adolescent and adult mice when they first acquire a conditional associative learning task. In a version of the human RL+WM task translated for rodents, mice were required to associate odor stimuli (from a set of 2 or 4 odors) with a left or right port to receive reward. Using logistic regression and computational models to analyze the first 200 trials per odor, we determined that mice used both incremental RL and stimulus-insensitive, one-back strategies to solve the task. While these one-back strategies may be a simple form of short-term or working memory, they did not approximate the boost to learning performance that has been observed in human participants using WM in a comparable task. Adolescent and adult mice also showed comparable performance, with no change in learning rate or softmax beta parameters with adolescent development and task experience. However, reliance on a one-back perseverative, win-stay strategy increased with development in males in both odor set sizes, but was not dependent on gonadal hormones. Our findings advance a simple conditional associative learning task and new models to enable the isolation and quantification of reinforcement learning alongside other strategies mice use while learning to associate stimuli with rewards within a single behavioral session. These data and methods can inform and aid comparative study of reinforcement learning across species.
计算模型表明，人类研究参与者使用快速工作记忆（WM）和增量强化学习（RL）（RL+WM）来解决简单的工具性学习任务，当刺激数量较少时依赖WM，当刺激数量超过WM容量时则补充RL。受这项工作的启发，我们研究了青春期和成年小鼠首次习得条件性联想学习任务时使用了哪些学习系统和策略。在为啮齿动物翻译的人类RL+WM任务版本中，小鼠需要将气味刺激（从一组2种或4种气味中选择）与左端口或右端口相关联以获得奖励。使用逻辑回归和计算模型分析每种气味的前200次试验，我们确定小鼠使用增量RL和对刺激不敏感的一步回溯策略来解决任务。虽然这些一步回溯策略可能是短期或工作记忆的一种简单形式，但它们并没有近似于在类似任务中使用WM的人类参与者所观察到的对学习表现的提升。青春期和成年小鼠也表现出相当的性能，随着青春期发育和任务经验的增加，学习率或softmaxβ参数没有变化。然而，在两种气味集大小中，雄性小鼠对一步回溯坚持性赢留策略的依赖随着发育而增加，但不依赖于性腺激素。我们的研究结果推进了一个简单的条件性联想学习任务和新模型，以能够在单个行为会话中学习将刺激与奖励相关联时，分离和量化强化学习以及小鼠使用的其他策略。这些数据和方法可以为跨物种强化学习的比较研究提供信息和帮助。
