Oita University, 700 Dannoharu, Oita, 870-1192, Japan.
Neural Netw. 2020 Dec;132:19-29. doi: 10.1016/j.neunet.2020.08.002. Epub 2020 Aug 13.
This paper addresses learning with exploration driven by chaotic internal dynamics of a neural network. Hoerzer et al. showed that a chaotic reservoir network (RN) can learn with exploration driven by external random noise and a sequential reward. In this paper, we demonstrate that a chaotic RN can learn without external noise because the output fluctuation originated from its internal chaotic dynamics functions as exploration. As learning progresses, the chaoticity decreases and the network can automatically switch from exploration mode to exploitation mode. Furthermore, the network can resume exploration when presented with a new situation. In addition, we found that even when the two parameters that influence the chaoticity are varied, learning performance always improves around the edge of chaos. From these results, we think that exploration is generated from internal chaotic dynamics, and exploitation appears in the process of forming attractors on the chaotic dynamics through learning. Consequently, exploration and exploitation are well-balanced around the edge of chaos, which leads to good learning performance.
本文探讨了由神经网络内部混沌动力学驱动的探索式学习。Hoerzer 等人表明，混沌储层网络（RN）可以在外部随机噪声和顺序奖励的驱动下进行探索式学习。在本文中，我们证明了混沌 RN 可以在没有外部噪声的情况下进行学习，因为其内部混沌动力学产生的输出波动可以作为探索。随着学习的进行，混沌度降低，网络可以自动从探索模式切换到利用模式。此外，当网络遇到新情况时，它可以恢复探索。此外，我们发现，即使两个影响混沌的参数发生变化，学习性能也总是在混沌边缘得到改善。从这些结果中，我们认为探索是由内部混沌动力学产生的，而利用则出现在通过学习在混沌动力学上形成吸引子的过程中。因此，探索和利用在混沌边缘达到良好的平衡，从而实现了良好的学习性能。
