Inter-university Semiconductor Research Center (ISRC) and Department of Electrical and Computer Engineering, Seoul National University, Seoul, 08826, Republic of Korea.
Sci Rep. 2020 Feb 26;10(1):3515. doi: 10.1038/s41598-020-60572-8.
Spiking neural networks (SNNs) are considered as the third generation of artificial neural networks, having the potential to improve the energy efficiency of conventional computing systems. Although the firing rate of a spiking neuron is an approximation of rectified linear unit (ReLU) activation in an analog-valued neural network (ANN), there remain many challenges to be overcome owing to differences in operation between ANNs and SNNs. Unlike actual biological and biophysical processes, various hardware implementations of neurons and SNNs do not allow the membrane potential to fall below the resting potential-in other words, neurons must allow the sub-resting membrane potential. Because there occur an excitatory post-synaptic potential (EPSP) as well as an inhibitory post-synaptic potential (IPSP), negatively valued synaptic weights in SNNs induce the sub-resting membrane potential at some time point. If a membrane is not allowed to hold the sub-resting potential, errors will accumulate over time, resulting in inaccurate inference operations. This phenomenon is not observed in ANNs given their use of only spatial synaptic integration, but it can cause serious performance degradation in SNNs. In this paper, we demonstrate the impact of the sub-resting membrane potential on accurate inference operations in SNNs. Moreover, several important considerations for a hardware SNN that can maintain the sub-resting membrane potential are discussed. All of the results in this paper indicate that it is essential for neurons to allow the sub-resting membrane potential in order to realize high-performance SNNs.
尖峰神经网络（SNN）被认为是第三代人工神经网络，有可能提高传统计算系统的能效。虽然尖峰神经元的发放率是模拟值神经网络（ANN）中整流线性单元（ReLU）激活的一种近似，但由于 ANN 和 SNN 之间的操作差异，仍有许多挑战需要克服。与实际的生物和生物物理过程不同，神经元和 SNN 的各种硬件实现不允许膜电位降至静息电位以下——换句话说，神经元必须允许亚静息膜电位。由于存在兴奋性突触后电位（EPSP）和抑制性突触后电位（IPSP），SNN 中的负值突触权重会在某些时刻诱导亚静息膜电位。如果不允许膜保持亚静息电位，误差会随时间累积，导致不准确的推断操作。由于 ANN 仅使用空间突触整合，因此不会观察到这种现象，但它会导致 SNN 的性能严重下降。在本文中，我们展示了亚静息膜电位对 SNN 中准确推断操作的影响。此外，还讨论了能够维持亚静息膜电位的硬件 SNN 的几个重要注意事项。本文中的所有结果都表明，为了实现高性能的 SNN，神经元允许亚静息膜电位是至关重要的。
