Yang Shuangming, Gao Tian, Wang Jiang, Deng Bin, Lansdell Benjamin, Linares-Barranco Bernabe
School of Electrical and Information Engineering, Tianjin University, Tianjin, China.
Department of Bioengineering, University of Pennsylvania, Philadelphia, PA, United States.
Front Neurosci. 2021 Feb 19;15:601109. doi: 10.3389/fnins.2021.601109. eCollection 2021.
A critical challenge in neuromorphic computing is to present computationally efficient algorithms of learning. When implementing gradient-based learning, error information must be routed through the network, such that each neuron knows its contribution to output, and thus how to adjust its weight. This is known as the credit assignment problem. Exactly implementing a solution like backpropagation involves weight sharing, which requires additional bandwidth and computations in a neuromorphic system. Instead, models of learning from neuroscience can provide inspiration for how to communicate error information efficiently, without weight sharing. Here we present a novel dendritic event-based processing (DEP) algorithm, using a two-compartment leaky integrate-and-fire neuron with partially segregated dendrites that effectively solves the credit assignment problem. In order to optimize the proposed algorithm, a dynamic fixed-point representation method and piecewise linear approximation approach are presented, while the synaptic events are binarized during learning. The presented optimization makes the proposed DEP algorithm very suitable for implementation in digital or mixed-signal neuromorphic hardware. The experimental results show that spiking representations can rapidly learn, achieving high performance by using the proposed DEP algorithm. We find the learning capability is affected by the degree of dendritic segregation, and the form of synaptic feedback connections. This study provides a bridge between the biological learning and neuromorphic learning, and is meaningful for the real-time applications in the field of artificial intelligence.
神经形态计算中的一个关键挑战是提出计算高效的学习算法。在实现基于梯度的学习时,误差信息必须在网络中进行路由,以便每个神经元都知道其对输出的贡献,进而知道如何调整其权重。这就是所谓的信用分配问题。精确实现类似反向传播的解决方案涉及权重共享,这在神经形态系统中需要额外的带宽和计算。相反,来自神经科学的学习模型可以为如何在不进行权重共享的情况下有效传达误差信息提供灵感。在这里,我们提出了一种新颖的基于树突事件的处理(DEP)算法,该算法使用具有部分隔离树突的双室泄漏积分发放神经元,有效地解决了信用分配问题。为了优化所提出的算法,提出了一种动态定点表示方法和分段线性近似方法,同时在学习过程中将突触事件进行二值化。所提出的优化使得所提出的DEP算法非常适合在数字或混合信号神经形态硬件中实现。实验结果表明,通过使用所提出的DEP算法,脉冲表示可以快速学习并实现高性能。我们发现学习能力受树突隔离程度和突触反馈连接形式的影响。这项研究为生物学习和神经形态学习之间架起了一座桥梁,对人工智能领域的实时应用具有重要意义。