Machine Learning Group, Department of Physics and Technology, University of Tromsø, 9019 Tromsø, Norway.
Department of Computer Science, College of Engineering, Mathematics and Physical Sciences, University of Exeter, Exeter EX4 4QF, United Kingdom.
Sci Rep. 2017 Mar 10;7:44037. doi: 10.1038/srep44037.
A recurrent neural network (RNN) is a universal approximator of dynamical systems, whose performance often depends on sensitive hyperparameters. Tuning them properly may be difficult and, typically, based on a trial-and-error approach. In this work, we adopt a graph-based framework to interpret and characterize internal dynamics of a class of RNNs called echo state networks (ESNs). We design principled unsupervised methods to derive hyperparameters configurations yielding maximal ESN performance, expressed in terms of prediction error and memory capacity. In particular, we propose to model time series generated by each neuron activations with a horizontal visibility graph, whose topological properties have been shown to be related to the underlying system dynamics. Successively, horizontal visibility graphs associated with all neurons become layers of a larger structure called a multiplex. We show that topological properties of such a multiplex reflect important features of ESN dynamics that can be used to guide the tuning of its hyperparamers. Results obtained on several benchmarks and a real-world dataset of telephone call data records show the effectiveness of the proposed methods.
递归神经网络（RNN）是动力系统的通用逼近器，其性能通常取决于敏感的超参数。正确调整它们可能很困难，通常基于试错法。在这项工作中，我们采用基于图的框架来解释和描述一类称为回声状态网络（ESN）的 RNN 的内部动力学。我们设计了有原则的无监督方法来推导超参数配置，以获得最大的 ESN 性能，以预测误差和记忆容量来表示。具体来说，我们建议使用水平可视性图来对每个神经元激活生成的时间序列进行建模，其拓扑性质已被证明与基础系统动力学有关。随后，与所有神经元相关的水平可视性图成为一个更大结构的层，称为复用。我们表明，这种复用的拓扑性质反映了 ESN 动力学的重要特征，可用于指导其超参数的调整。在几个基准测试和一个电话呼叫记录的真实数据集上获得的结果表明了所提出方法的有效性。
