Zoppo Gianluca, Marrone Francesco, Corinto Fernando
Department of Electronics, Politecnico di Torino, Turin, Italy.
Front Neurosci. 2020 Mar 24;14:240. doi: 10.3389/fnins.2020.00240. eCollection 2020.
Among the recent innovative technologies, memristor (memory-resistor) has attracted researchers attention as a fundamental computation element. It has been experimentally shown that memristive elements can emulate synaptic dynamics and are even capable of supporting spike timing dependent plasticity (STDP), an important adaptation rule that is gaining particular interest because of its simplicity and biological plausibility. The overall goal of this work is to provide a novel (theoretical) analog computing platform based on memristor devices and recurrent neural networks that exploits the memristor device physics to implement two variations of the backpropagation algorithm: recurrent backpropagation and equilibrium propagation. In the first learning technique, the use of memristor-based synaptic weights permits to propagate the error signals in the network by means of the nonlinear dynamics via an analog side network. This makes the processing non-digital and different from the current procedures. However, the necessity of a side analog network for the propagation of error derivatives makes this technique still highly biologically implausible. In order to solve this limitation, it is therefore proposed an alternative solution to the use of a side network by introducing a learning technique used for energy-based models: equilibrium propagation. Experimental results show that both approaches significantly outperform conventional architectures used for pattern reconstruction. Furthermore, due to the high suitability for VLSI implementation of the equilibrium propagation learning rule, additional results on the classification of the MNIST dataset are here reported.
在最近的创新技术中，忆阻器（记忆电阻器）作为一种基本的计算元件引起了研究人员的关注。实验表明，忆阻元件可以模拟突触动态，甚至能够支持尖峰时间依赖可塑性（STDP），这是一种重要的适应规则，因其简单性和生物学合理性而备受关注。这项工作的总体目标是提供一个基于忆阻器器件和递归神经网络的新型（理论）模拟计算平台，该平台利用忆阻器器件物理特性来实现反向传播算法的两种变体：递归反向传播和平衡传播。在第一种学习技术中，基于忆阻器的突触权重的使用允许通过模拟侧网络借助非线性动力学在网络中传播误差信号。这使得处理是非数字的，并且与当前程序不同。然而，需要一个侧模拟网络来传播误差导数使得该技术在生物学上仍然高度不可信。为了解决这个限制，因此通过引入用于基于能量模型的学习技术：平衡传播，提出了一种替代侧网络使用的解决方案。实验结果表明，这两种方法都显著优于用于模式重建的传统架构。此外，由于平衡传播学习规则对超大规模集成电路（VLSI）实现具有高度适用性，这里报告了关于MNIST数据集分类的其他结果。
