Kaiser Jacques, Mostafa Hesham, Neftci Emre
FZI Research Center for Information Technology, Karlsruhe, Germany.
Department of Bioengineering, University of California, San Diego, La Jolla, CA, United States.
Front Neurosci. 2020 May 12;14:424. doi: 10.3389/fnins.2020.00424. eCollection 2020.
A growing body of work underlines striking similarities between biological neural networks and recurrent, binary neural networks. A relatively smaller body of work, however, addresses the similarities between learning dynamics employed in deep artificial neural networks and synaptic plasticity in spiking neural networks. The challenge preventing this is largely caused by the discrepancy between the dynamical properties of synaptic plasticity and the requirements for gradient backpropagation. Learning algorithms that approximate gradient backpropagation using local error functions can overcome this challenge. Here, we introduce Deep Continuous Local Learning (DECOLLE), a spiking neural network equipped with local error functions for online learning with no memory overhead for computing gradients. DECOLLE is capable of learning deep spatio temporal representations from spikes relying solely on local information, making it compatible with neurobiology and neuromorphic hardware. Synaptic plasticity rules are derived systematically from user-defined cost functions and neural dynamics by leveraging existing autodifferentiation methods of machine learning frameworks. We benchmark our approach on the event-based neuromorphic dataset N-MNIST and DvsGesture, on which DECOLLE performs comparably to the state-of-the-art. DECOLLE networks provide continuously learning machines that are relevant to biology and supportive of event-based, low-power computer vision architectures matching the accuracies of conventional computers on tasks where temporal precision and speed are essential.
越来越多的研究工作强调了生物神经网络与循环二元神经网络之间惊人的相似性。然而，相对较少的研究工作涉及深度人工神经网络中使用的学习动态与脉冲神经网络中突触可塑性之间的相似性。阻碍这一研究的挑战主要是由突触可塑性的动态特性与梯度反向传播的要求之间的差异造成的。使用局部误差函数近似梯度反向传播的学习算法可以克服这一挑战。在这里，我们介绍深度连续局部学习（DECOLLE），这是一种脉冲神经网络，配备了局部误差函数用于在线学习，且计算梯度时没有内存开销。DECOLLE能够仅依靠局部信息从脉冲中学习深度时空表示，使其与神经生物学和神经形态硬件兼容。通过利用机器学习框架现有的自动微分方法，从用户定义的成本函数和神经动力学中系统地推导突触可塑性规则。我们在基于事件的神经形态数据集N-MNIST和DvsGesture上对我们的方法进行基准测试，在这些数据集上DECOLLE的表现与当前最先进的方法相当。DECOLLE网络提供了持续学习的机器，这些机器与生物学相关，并支持基于事件的低功耗计算机视觉架构，在时间精度和速度至关重要的任务上与传统计算机的精度相匹配。
