Department of Computer Science, University of Pisa, Largo B. Pontecorvo 3, Pisa, Italy.
Department of Computer Science, University of Pisa, Largo B. Pontecorvo 3, Pisa, Italy.
Neural Netw. 2018 Dec;108:33-47. doi: 10.1016/j.neunet.2018.08.002. Epub 2018 Aug 8.
In this paper, we provide a novel approach to the architectural design of deep Recurrent Neural Networks using signal frequency analysis. In particular, focusing on the Reservoir Computing framework and inspired by the principles related to the inherent effect of layering, we address a fundamental open issue in deep learning, namely the question of how to establish the number of layers in recurrent architectures in the form of deep echo state networks (DeepESNs). The proposed method is first analyzed and refined on a controlled scenario and then it is experimentally assessed on challenging real-world tasks. The achieved results also show the ability of properly designed DeepESNs to outperform RC approaches on a speech recognition task, and to compete with the state-of-the-art in time-series prediction on polyphonic music tasks.
在本文中，我们提出了一种基于信号频率分析的深度递归神经网络架构设计的新方法。具体来说，我们聚焦于 Reservoir Computing 框架，并受分层固有效应原理的启发，解决了深度学习中的一个基本开放性问题，即如何以深度回声状态网络（DeepESN）的形式确定递归架构中的层数。所提出的方法首先在受控场景中进行分析和改进，然后在具有挑战性的真实任务中进行实验评估。所得到的结果还表明，适当设计的 DeepESN 能够在语音识别任务上优于 RC 方法，并在多音音乐任务的时间序列预测方面与最先进的方法相竞争。
