Brain Science Institute, RIKEN, 2-1 Hirosawa, Wako-shi, Saitama, 351-0198, Japan.
Neural Netw. 2010 Jun;23(5):625-38. doi: 10.1016/j.neunet.2009.12.006. Epub 2009 Dec 23.
This study shows that a mixture of RNN experts model can acquire the ability to generate sequences that are combination of multiple primitive patterns by means of self-organizing chaos. By training the model, each expert learns a primitive sequence pattern, and a gating network learns to imitate stochastic switching of the multiple primitives via chaotic dynamics, utilizing a sensitive dependence on initial conditions. As a demonstration, we present a numerical simulation in which the model learns Markov chain switching among some Lissajous curves by chaotic dynamics. Our analysis shows that by using a sufficient amount of training data, balanced with the network memory capacity, it is possible to satisfy the conditions for embedding the target stochastic sequences into a chaotic dynamical system. It is also shown that reconstruction of a stochastic time series by a chaotic model can be stabilized by adding a negligible amount of noise to the dynamics of the model.
本研究表明，通过自组织混沌，RNN 专家混合模型可以获得生成由多个基元模式组合而成的序列的能力。通过对模型进行训练，每个专家都学习到一个基元序列模式，而门控网络则通过混沌动力学来学习模仿多个基元的随机切换，利用对初始条件的敏感依赖性。作为一个演示，我们提出了一个数值模拟，其中模型通过混沌动力学学习一些利萨如曲线之间的马尔可夫链切换。我们的分析表明，通过使用足够数量的训练数据，与网络记忆容量相平衡，可以满足将目标随机序列嵌入混沌动力系统的条件。还表明，通过向模型的动力学添加可忽略的噪声，可以稳定混沌模型对随机时间序列的重构。
