Vodret Michele
<a href="https://ror.org/03xjwb503">Université Paris-Saclay</a>, CentraleSupélec, Laboratoire de Mathématiques et Informatique pour la Complexité et les Systèmes, 91192 Gif-sur-Yvette, France.
Phys Rev E. 2024 Jul;110(1-1):014304. doi: 10.1103/PhysRevE.110.014304.
The relationship between time irreversibility in neuronal dynamics and cognitive effort is a subject of growing interest in the scientific literature. Although correlations between proxies of both concepts have been experimentally observed, the underlying precise linkage between them remains elusive. Here we investigate the case of learning in decision-making tasks; we do so by introducing a thermodynamically grounded metric-inspired by Landauer's principle-which connects time-irreversible information processing to energy consumption. Equipped with this metric, we investigate the role of macroscopic time-reversal symmetry breaking in belief dynamics for the case of an agent with finite sensitivity while performing a static two-armed bandit task-a standard setup in cognitive neuroscience. To gain insights into the belief dynamics, we analogize it to the dynamics of an active particle subject to state-dependent noise and living in a two-dimensional space. This mapping allows an analytical description of learning-induced biases. We deeply explore the case of Q-learning with forgetting the nonchosen option. In this case, learning-induced risk aversion is formally equivalent to standard thermophoresis, i.e., the net motion towards low-temperature regions. Finally, we quantify the irreversibility of belief dynamics in the steady state for different bandit configurations, sensitivity levels, and exploitative behavior. We found a strong correlation in high-sensitivity learning between heightened irreversibility in belief dynamics and improved decision-making outcomes. Notably, as the task's difficulty increases, a greater degree of irreversibility in belief dynamics becomes necessary for having superior performances; this explicitly unravels a plausible connection between time irreversibility and cognitive effort. In conclusion, our investigation reveals that irreversibility in belief dynamics bridges out-of-equilibrium statistical physics concepts and cognitive neuroscience. In decision-making contexts, this perspective offers insights into the notion of cognitive effort, suggesting a potential mechanism driving the evolution of living systems toward out-of-equilibrium structures.
神经元动力学中的时间不可逆性与认知努力之间的关系是科学文献中越来越受关注的主题。尽管这两个概念的代理指标之间的相关性已通过实验观察到，但它们之间潜在的精确联系仍然难以捉摸。在这里，我们研究决策任务中的学习情况；我们通过引入一种受兰道尔原理启发的基于热力学的度量来进行研究，该度量将时间不可逆的信息处理与能量消耗联系起来。借助这个度量，我们研究了宏观时间反演对称性破缺在信念动力学中的作用，该信念动力学是针对在执行静态双臂赌博任务（认知神经科学中的标准设置）时具有有限敏感性的主体而言的。为了深入了解信念动力学，我们将其类比为一个处于二维空间且受状态依赖噪声影响的活性粒子的动力学。这种映射允许对学习引起的偏差进行解析描述。我们深入探讨了遗忘未选择选项的Q学习情况。在这种情况下，学习引起的风险规避在形式上等同于标准的热泳，即向低温区域的净运动。最后，我们量化了不同赌博配置、敏感性水平和探索行为下稳态信念动力学的不可逆性。我们发现，在高敏感性学习中，信念动力学中增强的不可逆性与改善的决策结果之间存在很强的相关性。值得注意的是，随着任务难度的增加，信念动力学中更大程度的不可逆性对于获得卓越表现变得必要；这明确揭示了时间不可逆性与认知努力之间的合理联系。总之，我们的研究表明，信念动力学中的不可逆性架起了非平衡统计物理概念与认知神经科学之间的桥梁。在决策背景下，这种观点为认知努力的概念提供了见解，暗示了一种驱动生命系统向非平衡结构进化的潜在机制。
