Division of Humanities and Social Sciences, Caltech, Pasadena, CA, USA.
Division of Humanities and Social Sciences, Caltech, Pasadena, CA, USA.
Neuron. 2022 Aug 17;110(16):2691-2702.e8. doi: 10.1016/j.neuron.2022.05.025. Epub 2022 Jul 8.
Both novelty and uncertainty are potent features guiding exploration; however, they are often experimentally conflated, and an understanding of how they interact to regulate the balance between exploration and exploitation has proved elusive. Using a task designed to decouple the influence of novelty and uncertainty, we identify separable mechanisms through which exploration is directed. We show that uncertainty-directed exploration is sensitive to the prospective benefit offered by new information, whereas novelty-directed exploration is maintained regardless of its potential advantage. Using a computational framework in conjunction with fMRI, we show that uncertainty-directed choice is rooted in an adaptive bias indexing the prospective utility of exploration. In contrast, novelty persistently promotes exploration by optimistically inflating reward expectations while simultaneously dampening uncertainty signals. Our results identify separable neural substrates charged with balancing the explore/exploit trade-off to foster a manageable decomposition of an otherwise intractable problem.
新颖性和不确定性都是指导探索的有力特征；然而，它们通常在实验中被混淆，并且理解它们如何相互作用以调节探索和利用之间的平衡一直难以捉摸。使用一项旨在分离新颖性和不确定性影响的任务，我们确定了探索的可分离机制。我们表明，不确定性导向的探索对新信息提供的预期收益敏感，而新颖性导向的探索则不受其潜在优势的影响。我们使用计算框架结合 fMRI 表明，不确定性导向的选择源于一个自适应偏差，该偏差索引了探索的预期效用。相比之下，新颖性通过乐观地夸大奖励预期，同时抑制不确定性信号，持续促进探索。我们的结果确定了可分离的神经基质，负责平衡探索/利用的权衡，以促进对原本难以解决的问题进行可管理的分解。
