Schwöbel Sarah, Kiebel Stefan, Marković Dimitrije
Department of Psychology, Technische Universität Dresden, Dresden 01187, Germany
Neural Comput. 2018 Sep;30(9):2530-2567. doi: 10.1162/neco_a_01108. Epub 2018 Jun 27.
When modeling goal-directed behavior in the presence of various sources of uncertainty, planning can be described as an inference process. A solution to the problem of planning as inference was previously proposed in the active inference framework in the form of an approximate inference scheme based on variational free energy. However, this approximate scheme was based on the mean-field approximation, which assumes statistical independence of hidden variables and is known to show overconfidence and may converge to local minima of the free energy. To better capture the spatiotemporal properties of an environment, we reformulated the approximate inference process using the so-called Bethe approximation. Importantly, the Bethe approximation allows for representation of pairwise statistical dependencies. Under these assumptions, the minimizer of the variational free energy corresponds to the belief propagation algorithm, commonly used in machine learning. To illustrate the differences between the mean-field approximation and the Bethe approximation, we have simulated agent behavior in a simple goal-reaching task with different types of uncertainties. Overall, the Bethe agent achieves higher success rates in reaching goal states. We relate the better performance of the Bethe agent to more accurate predictions about the consequences of its own actions. Consequently, active inference based on the Bethe approximation extends the application range of active inference to more complex behavioral tasks.
在存在各种不确定性来源的情况下对目标导向行为进行建模时，规划可被描述为一个推理过程。先前在主动推理框架中以基于变分自由能的近似推理方案的形式提出了一种将规划问题作为推理的解决方案。然而，这种近似方案基于平均场近似，它假定隐藏变量的统计独立性，并且已知会表现出过度自信，可能会收敛到自由能的局部最小值。为了更好地捕捉环境的时空特性，我们使用所谓的贝叶斯近似重新制定了近似推理过程。重要的是，贝叶斯近似允许表示成对的统计依赖性。在这些假设下，变分自由能的极小值对应于机器学习中常用的信念传播算法。为了说明平均场近似和贝叶斯近似之间的差异，我们在具有不同类型不确定性的简单目标达成任务中模拟了智能体行为。总体而言，贝叶斯智能体在达到目标状态方面实现了更高的成功率。我们将贝叶斯智能体的更好性能与对自身行动后果的更准确预测联系起来。因此，基于贝叶斯近似的主动推理将主动推理的应用范围扩展到更复杂的行为任务。
