Sakemi Yusuke, Yamamoto Kakei, Hosomi Takeo, Aihara Kazuyuki
Research Center for Mathematical Engineering, Chiba Institute of Technology, Narashino, Japan.
International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo, Tokyo, Japan.
Sci Rep. 2023 Dec 21;13(1):22897. doi: 10.1038/s41598-023-50201-5.
The training of multilayer spiking neural networks (SNNs) using the error backpropagation algorithm has made significant progress in recent years. Among the various training schemes, the error backpropagation method that directly uses the firing time of neurons has attracted considerable attention because it can realize ideal temporal coding. This method uses time-to-first-spike (TTFS) coding, in which each neuron fires at most once, and this restriction on the number of firings enables information to be processed at a very low firing frequency. This low firing frequency increases the energy efficiency of information processing in SNNs. However, only an upper limit has been provided for TTFS-coded SNNs, and the information-processing capability of SNNs at lower firing frequencies has not been fully investigated. In this paper, we propose two spike-timing-based sparse-firing (SSR) regularization methods to further reduce the firing frequency of TTFS-coded SNNs. Both methods are characterized by the fact that they only require information about the firing timing and associated weights. The effects of these regularization methods were investigated on the MNIST, Fashion-MNIST, and CIFAR-10 datasets using multilayer perceptron networks and convolutional neural network structures.
近年来，使用误差反向传播算法训练多层脉冲神经网络（SNN）取得了显著进展。在各种训练方案中，直接使用神经元发放时间的误差反向传播方法因其能够实现理想的时间编码而备受关注。该方法采用首次发放时间（TTFS）编码，其中每个神经元最多发放一次，这种对发放次数的限制使得信息能够以非常低的发放频率进行处理。这种低发放频率提高了SNN中信息处理的能量效率。然而，对于基于TTFS编码的SNN仅给出了一个上限，并且SNN在较低发放频率下的信息处理能力尚未得到充分研究。在本文中，我们提出了两种基于脉冲时间的稀疏发放（SSR）正则化方法，以进一步降低基于TTFS编码的SNN的发放频率。这两种方法的特点是它们只需要关于发放时间和相关权重的信息。使用多层感知器网络和卷积神经网络结构，在MNIST、Fashion-MNIST和CIFAR-10数据集上研究了这些正则化方法的效果。
