Koudahl Magnus T, Kouw Wouter M, de Vries Bert
Department of Electrical Engineering, Eindhoven University of Technology, 5612 AZ Eindhoven, The Netherlands.
GN Hearing, JF Kennedylaan 2, 5612 AB Eindhoven, The Netherlands.
Entropy (Basel). 2021 Nov 24;23(12):1565. doi: 10.3390/e23121565.
Active Inference (AIF) is a framework that can be used both to describe information processing in naturally intelligent systems, such as the human brain, and to design synthetic intelligent systems (agents). In this paper we show that Expected Free Energy (EFE) minimisation, a core feature of the framework, does not lead to purposeful explorative behaviour in linear Gaussian dynamical systems. We provide a simple proof that, due to the specific construction used for the EFE, the terms responsible for the exploratory (epistemic) drive become constant in the case of linear Gaussian systems. This renders AIF equivalent to KL control. From a theoretical point of view this is an interesting result since it is generally assumed that EFE minimisation will always introduce an exploratory drive in AIF agents. While the full EFE objective does not lead to exploration in linear Gaussian dynamical systems, the principles of its construction can still be used to design objectives that include an epistemic drive. We provide an in-depth analysis of the mechanics behind the epistemic drive of AIF agents and show how to design objectives for linear Gaussian dynamical systems that do include an epistemic drive. Concretely, we show that focusing solely on epistemics and dispensing with goal-directed terms leads to a form of maximum entropy exploration that is heavily dependent on the type of control signals driving the system. Additive controls do not permit such exploration. From a practical point of view this is an important result since linear Gaussian dynamical systems with additive controls are an extensively used model class, encompassing for instance Linear Quadratic Gaussian controllers. On the other hand, linear Gaussian dynamical systems driven by multiplicative controls such as switching transition matrices do permit an exploratory drive.
主动推理（AIF）是一个框架，既可以用于描述自然智能系统（如人类大脑）中的信息处理，也可以用于设计合成智能系统（智能体）。在本文中，我们表明，该框架的核心特征——期望自由能量（EFE）最小化，并不会在线性高斯动力系统中导致有目的的探索行为。我们提供了一个简单的证明，由于用于EFE的特定构造，在高斯线性系统中，负责探索（认知）驱动的项会变得恒定。这使得AIF等同于KL控制。从理论角度来看，这是一个有趣的结果，因为通常认为EFE最小化总会在AIF智能体中引入探索驱动。虽然完整的EFE目标不会在线性高斯动力系统中导致探索，但它的构造原理仍可用于设计包含认知驱动的目标。我们对AIF智能体认知驱动背后的机制进行了深入分析，并展示了如何为确实包含认知驱动的线性高斯动力系统设计目标。具体而言，我们表明，仅关注认知并摒弃目标导向项会导致一种严重依赖驱动系统的控制信号类型的最大熵探索形式。加法控制不允许这种探索。从实际角度来看，这是一个重要的结果，因为具有加法控制的线性高斯动力系统是一个广泛使用的模型类别，例如包括线性二次高斯控制器。另一方面，由乘法控制（如切换转移矩阵）驱动的线性高斯动力系统确实允许探索驱动。
