Jiang Chunming, Zhang Yilei
Department of Mechanical Engineering, University of Canterbury, Canterbury CT2 7NX, New Zealand
Neural Comput. 2023 Aug 7;35(9):1593-1608. doi: 10.1162/neco_a_01604.
Spiking neural networks (SNNs) are receiving increasing attention due to their low power consumption and strong bioplausibility. Optimization of SNNs is a challenging task. Two main methods, artificial neural network (ANN)-to-SNN conversion and spike-based backpropagation (BP), both have advantages and limitations. ANN-to-SNN conversion requires a long inference time to approximate the accuracy of ANN, thus diminishing the benefits of SNN. With spike-based BP, training high-precision SNNs typically consumes dozens of times more computational resources and time than their ANN counterparts. In this letter, we propose a novel SNN training approach that combines the benefits of the two methods. We first train a single-step SNN(T = 1) by approximating the neural potential distribution with random noise, then convert the single-step SNN(T = 1) to a multistep SNN(T = N) losslessly. The introduction of gaussian distributed noise leads to a significant gain in accuracy after conversion. The results show that our method considerably reduces the training and inference times of SNNs while maintaining their high accuracy. Compared to the previous two methods, ours can reduce training time by 65% to 75% and achieves more than 100 times faster inference speed. We also argue that the neuron model augmented with noise makes it more bioplausible.
脉冲神经网络（SNN）因其低功耗和强大的生物合理性而受到越来越多的关注。SNN的优化是一项具有挑战性的任务。两种主要方法，即人工神经网络（ANN）到SNN的转换和基于脉冲的反向传播（BP），都有优点和局限性。ANN到SNN的转换需要很长的推理时间来逼近ANN的精度，从而削弱了SNN的优势。使用基于脉冲的BP，训练高精度的SNN通常比训练ANN消耗数十倍的计算资源和时间。在这封信中，我们提出了一种新颖的SNN训练方法，它结合了这两种方法的优点。我们首先通过用随机噪声逼近神经电位分布来训练单步SNN（T = 1），然后将单步SNN（T = 1）无损地转换为多步SNN（T = N）。高斯分布噪声的引入在转换后带来了显著的精度提升。结果表明，我们的方法在保持SNN高精度的同时，大大减少了其训练和推理时间。与前两种方法相比，我们的方法可以将训练时间减少65%到75%，推理速度提高100倍以上。我们还认为，用噪声增强的神经元模型使其更具生物合理性。
