Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
Neural Netw. 2012 Nov;35:1-9. doi: 10.1016/j.neunet.2012.07.005. Epub 2012 Jul 23.
An echo state network (ESN) consists of a large, randomly connected neural network, the reservoir, which is driven by an input signal and projects to output units. During training, only the connections from the reservoir to these output units are learned. A key requisite for output-only training is the echo state property (ESP), which means that the effect of initial conditions should vanish as time passes. In this paper, we use analytical examples to show that a widely used criterion for the ESP, the spectral radius of the weight matrix being smaller than unity, is not sufficient to satisfy the echo state property. We obtain these examples by investigating local bifurcation properties of the standard ESNs. Moreover, we provide new sufficient conditions for the echo state property of standard sigmoid and leaky integrator ESNs. We furthermore suggest an improved technical definition of the echo state property, and discuss what practicians should (and should not) observe when they optimize their reservoirs for specific tasks.
回声状态网络 (ESN) 由一个大型的、随机连接的神经网络——储层组成，储层由输入信号驱动，并向输出单元投影。在训练过程中，仅从储层到这些输出单元的连接被学习。仅使用输出进行训练的一个关键要求是回声状态属性 (ESP)，这意味着随着时间的推移，初始条件的影响应该消失。在本文中，我们使用分析示例表明，ESP 的一个常用标准，即权重矩阵的谱半径小于 1，不足以满足回声状态属性。我们通过研究标准 ESN 的局部分岔性质获得了这些示例。此外，我们为标准的 sigmoid 和漏积分器 ESN 的回声状态属性提供了新的充分条件。我们还建议了回声状态属性的改进技术定义，并讨论了当从业人员为特定任务优化其储层时，他们应该（和不应该）观察到什么。
