Faraji Mohammadjavad, Preuschoff Kerstin, Gerstner Wulfram
School of Computer and Communication Sciences and School of Life Sciences, Brain Mind Institute, École Polytechnique Fédéral de Lausanne, 1015 Lausanne EPFL, Switzerland
Geneva Finance Research Institute and Center for Affective Sciences, University of Geneva, 1211 Geneva, Switzerland
Neural Comput. 2018 Jan;30(1):34-83. doi: 10.1162/neco_a_01025. Epub 2017 Oct 24.
Surprise describes a range of phenomena from unexpected events to behavioral responses. We propose a novel measure of surprise and use it for surprise-driven learning. Our surprise measure takes into account data likelihood as well as the degree of commitment to a belief via the entropy of the belief distribution. We find that surprise-minimizing learning dynamically adjusts the balance between new and old information without the need of knowledge about the temporal statistics of the environment. We apply our framework to a dynamic decision-making task and a maze exploration task. Our surprise-minimizing framework is suitable for learning in complex environments, even if the environment undergoes gradual or sudden changes, and it could eventually provide a framework to study the behavior of humans and animals as they encounter surprising events.
惊奇描述了从意外事件到行为反应的一系列现象。我们提出了一种新颖的惊奇度量方法，并将其用于基于惊奇的学习。我们的惊奇度量方法既考虑了数据似然性，也通过信念分布的熵考虑了对一种信念的坚持程度。我们发现，使惊奇最小化的学习能够动态地调整新旧信息之间的平衡，而无需了解环境的时间统计信息。我们将我们的框架应用于一个动态决策任务和一个迷宫探索任务。我们的使惊奇最小化的框架适用于在复杂环境中学习，即使环境经历逐渐或突然的变化，并且它最终可能提供一个框架来研究人类和动物在遇到令人惊奇的事件时的行为。
