Tokyo Research Center, Aisin Corporation, Akihabara Daibiru 7F 1-18-13, Sotokanda, Chiyoda-ku, Tokyo 101-0021, Japan.
AISIN SOFTWARE Co., Ltd., Advance Square Kariya 7F 1-1-1, Aioicho, Kariya 448-0027, Aichi, Japan.
Sensors (Basel). 2022 Apr 8;22(8):2876. doi: 10.3390/s22082876.
Biologically inspired spiking neural networks (SNNs) are widely used to realize ultralow-power energy consumption. However, deep SNNs are not easy to train due to the excessive firing of spiking neurons in the hidden layers. To tackle this problem, we propose a novel but simple normalization technique called postsynaptic potential normalization. This normalization removes the subtraction term from the standard normalization and uses the second raw moment instead of the variance as the division term. The spike firing can be controlled, enabling the training to proceed appropriately, by conducting this simple normalization to the postsynaptic potential. The experimental results show that SNNs with our normalization outperformed other models using other normalizations. Furthermore, through the pre-activation residual blocks, the proposed model can train with more than 100 layers without other special techniques dedicated to SNNs.
受生物启发的尖峰神经网络 (SNN) 被广泛用于实现超低功耗能耗。然而，由于隐藏层中尖峰神经元的过度激发，深度 SNN 不易训练。为了解决这个问题，我们提出了一种新颖但简单的归一化技术，称为突触后电位归一化。这种归一化从标准归一化中删除减法项，并使用二阶原始矩而不是方差作为除法项。通过对突触后电位进行这种简单的归一化，可以控制尖峰放电，使训练能够适当进行。实验结果表明，使用我们的归一化的 SNN 优于使用其他归一化的其他模型。此外，通过预激活残差块，所提出的模型可以在没有其他专门用于 SNN 的特殊技术的情况下训练超过 100 层。
