Isomura Takuya
Laboratory for Neural Computation and Adaptation, RIKEN Center for Brain Science, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan.
Entropy (Basel). 2018 Jul 7;20(7):512. doi: 10.3390/e20070512.
The mutual information between the state of a neural network and the state of the external world represents the amount of information stored in the neural network that is associated with the external world. In contrast, the surprise of the sensory input indicates the unpredictability of the current input. In other words, this is a measure of inference ability, and an upper bound of the surprise is known as the variational free energy. According to the free-energy principle (FEP), a neural network continuously minimizes the free energy to perceive the external world. For the survival of animals, inference ability is considered to be more important than simply memorized information. In this study, the free energy is shown to represent the gap between the amount of information stored in the neural network and that available for inference. This concept involves both the FEP and the infomax principle, and will be a useful measure for quantifying the amount of information available for inference.
神经网络的状态与外部世界的状态之间的互信息表示存储在神经网络中与外部世界相关联的信息量。相比之下，感官输入的意外程度表明当前输入的不可预测性。换句话说，这是一种推理能力的度量，意外程度的上限被称为变分自由能。根据自由能原理（FEP），神经网络不断最小化自由能以感知外部世界。对于动物的生存而言，推理能力被认为比单纯记忆的信息更为重要。在本研究中，自由能被证明代表了存储在神经网络中的信息量与可用于推理的信息量之间的差距。这一概念涉及自由能原理和信息最大化原理，将成为量化可用于推理的信息量的有用度量。
