Laurent Patryk A
Department of Psychological and Brain Sciences, The Johns Hopkins University, 136 Ames Hall, 3400 N. Charles St, Baltimore MD, 21218, Telephone: +1-410-929-2562.
Cognit Comput. 2013 Mar 1;5(1):152-160. doi: 10.1007/s12559-012-9178-8.
Decision-making often requires taking into consideration immediate gains as well as delayed rewards. Studies of behavior have established that anticipated rewards are discounted according to a decreasing hyperbolic function. Although mathematical explanations for reward delay discounting have been offered, little has been proposed in terms of neural network mechanisms underlying discounting. There has been much recent interest in the potential role of the hippocampus. Here we demonstrate that a previously-established neural network model of hippocampal region CA3 contains a mechanism that could explain discounting in downstream reward-prediction systems (e.g., basal ganglia). As part of its normal function, the model forms codes for stimuli that are similar to future, predicted stimuli. This similarity provides a means for reward predictions associated with future stimuli to influence current decision-making. Simulations show that this "predictive similarity" decreases as the stimuli are separated in time, at a rate that is consistent with hyperbolic discounting.
决策通常需要考虑即时收益以及延迟奖励。行为研究已经证实，预期奖励会根据递减的双曲线函数进行折扣。尽管已经给出了奖励延迟折扣的数学解释，但对于折扣背后的神经网络机制却鲜有提及。最近，海马体的潜在作用备受关注。在此，我们证明，先前建立的海马体CA3区神经网络模型包含一种机制，该机制可以解释下游奖励预测系统（如基底神经节）中的折扣现象。作为其正常功能的一部分，该模型会形成与未来预测刺激相似的刺激编码。这种相似性为与未来刺激相关的奖励预测影响当前决策提供了一种方式。模拟结果表明，随着刺激在时间上的间隔增加，这种“预测相似性”会降低，其速率与双曲线折扣一致。
