IEEE Trans Neural Netw Learn Syst. 2020 Apr;31(4):1363-1374. doi: 10.1109/TNNLS.2019.2919903. Epub 2019 Jun 24.
Existing synaptic plasticity rules for optimizing the connections between neurons within the reservoir of echo state networks (ESNs) remain to be global in that the same type of plasticity rule with the same parameters is applied to all neurons. However, this is biologically implausible and practically inflexible for learning the structures in the input signals, thereby limiting the learning performance of ESNs. In this paper, we propose to use local plasticity rules that allow different neurons to use different types of plasticity rules and different parameters, which are achieved by optimizing the parameters of the local plasticity rules using the evolution strategy (ES) with covariance matrix adaptation (CMA-ES). We show that evolving neural plasticity will result in a synergistic learning of different plasticity rules, which plays an important role in improving the learning performance. Meanwhile, we show that the local plasticity rules can effectively alleviate synaptic interferences in learning the structure in sensory inputs. The proposed local plasticity rules are compared with a number of the state-of-the-art ESN models and the canonical ESN using a global plasticity rule on a set of widely used prediction and classification benchmark problems to demonstrate its competitive learning performance.
现有的用于优化回声状态网络 (ESN) 中神经元之间连接的突触可塑性规则仍然是全局的，即对所有神经元应用相同类型的具有相同参数的可塑性规则。然而，这在生物学上是不可信的，在学习输入信号的结构方面也不灵活，从而限制了 ESN 的学习性能。在本文中，我们提出使用局部可塑性规则，允许不同的神经元使用不同类型的可塑性规则和不同的参数，这是通过使用协方差矩阵自适应进化策略 (CMA-ES) 优化局部可塑性规则的参数来实现的。我们表明，进化神经可塑性将导致不同可塑性规则的协同学习，这对于提高学习性能起着重要作用。同时，我们表明局部可塑性规则可以有效地减轻学习感官输入结构时的突触干扰。所提出的局部可塑性规则与一组广泛使用的预测和分类基准问题上的许多最先进的 ESN 模型和使用全局可塑性规则的标准 ESN 进行了比较，以证明其具有竞争力的学习性能。
