University of Kent, School of Computing, Canterbury CT2 7NZ, U.K.
University of Birmingham, School of Psychology, Birmingham B15 2TT, U.K., and University of Kent, School of Computing, Canterbury CT2 7NZ, U.K.
Neural Comput. 2021 Sep 16;33(10):2762-2826. doi: 10.1162/neco_a_01422.
Active inference is a state-of-the-art framework in neuroscience that offers a unified theory of brain function. It is also proposed as a framework for planning in AI. Unfortunately, the complex mathematics required to create new models can impede application of active inference in neuroscience and AI research. This letter addresses this problem by providing a complete mathematical treatment of the active inference framework in discrete time and state spaces and the derivation of the update equations for any new model. We leverage the theoretical connection between active inference and variational message passing as described by John Winn and Christopher M. Bishop in 2005. Since variational message passing is a well-defined methodology for deriving Bayesian belief update equations, this letter opens the door to advanced generative models for active inference. We show that using a fully factorized variational distribution simplifies the expected free energy, which furnishes priors over policies so that agents seek unambiguous states. Finally, we consider future extensions that support deep tree searches for sequential policy optimization based on structure learning and belief propagation.
主动推理是神经科学领域的一种最新框架，它提供了大脑功能的统一理论。它也被提议作为人工智能规划的框架。不幸的是，创建新模型所需的复杂数学运算可能会阻碍主动推理在神经科学和人工智能研究中的应用。这封信通过提供离散时间和状态空间中主动推理框架的完整数学处理以及任何新模型的更新方程的推导来解决这个问题。我们利用了 John Winn 和 Christopher M. Bishop 在 2005 年描述的主动推理和变分信息传递之间的理论联系。由于变分信息传递是为推导贝叶斯信念更新方程而定义的一种方法，因此这封信为主动推理的高级生成模型打开了大门。我们表明，使用完全因子化的变分分布可以简化期望自由能，从而为策略提供先验信息，以便代理寻找明确的状态。最后，我们考虑了未来的扩展，这些扩展支持基于结构学习和信念传播的用于顺序策略优化的深度树搜索。
