Hasssan Ahmed, Meng Jian, Anupreetham Anupreetham, Seo Jae-Sun
School of Electrical and Computer Engineering, Cornell Tech, New York, NY, United States.
Front Neurosci. 2024 Sep 4;18:1440000. doi: 10.3389/fnins.2024.1440000. eCollection 2024.
Spiking neural networks (SNNs) have received increasing attention due to their high biological plausibility and energy efficiency. The binary spike-based information propagation enables efficient sparse computation in event-based and static computer vision applications. However, the weight precision and especially the membrane potential precision remain as high-precision values (e.g., 32 bits) in state-of-the-art SNN algorithms. Each neuron in an SNN stores the membrane potential over time and typically updates its value in every time step. Such frequent read/write operations of high-precision membrane potential incur storage and memory access overhead in SNNs, which undermines the SNNs' compatibility with resource-constrained hardware. To resolve this inefficiency, prior works have explored the time step reduction and low-precision representation of membrane potential at a limited scale and reported significant accuracy drops. Furthermore, while recent advances in on-device AI present pruning and quantization optimization with different architectures and datasets, simultaneous pruning with quantization is highly under-explored in SNNs. In this work, we present , a fully-quantized spiking neural network with , enabling the end-to-end low precision with significantly reduced operations on SNN. First, we propose an integer-only quantization scheme for the membrane potential with a stacked surrogate gradient function, a simple-yet-effective method that enables the smooth learning process of quantized SNN training. Second, we implement spatial-channel pruning with membrane potential prior, toward reducing the layer-wise computational complexity, and floating-point operations (FLOPs) in SNNs. Finally, to further improve the accuracy of low-precision and sparse SNN, we propose a self-adaptive learnable potential threshold for SNN training. Equipped with high biological adaptiveness, minimal computations, and memory utilization, SpQuant-SNN achieves state-of-the-art performance across multiple SNN models for both event-based and static image datasets, including both image classification and object detection tasks. The proposed SpQuant-SNN achieved up to 13× memory reduction and >4.7× FLOPs reduction with < 1.8% accuracy degradation for both classification and object detection tasks, compared to the SOTA baseline.
脉冲神经网络（SNN）因其高度的生物合理性和能源效率而受到越来越多的关注。基于二进制脉冲的信息传播在基于事件的和静态的计算机视觉应用中实现了高效的稀疏计算。然而，在当前最先进的SNN算法中，权重精度，尤其是膜电位精度仍保持为高精度值（例如32位）。SNN中的每个神经元会随时间存储膜电位，并通常在每个时间步更新其值。这种对高精度膜电位的频繁读/写操作在SNN中会产生存储和内存访问开销，这削弱了SNN与资源受限硬件的兼容性。为了解决这种低效率问题，先前的工作在有限规模上探索了减少时间步长和膜电位的低精度表示，并报告了显著的精度下降。此外，虽然设备端人工智能的最新进展提出了针对不同架构和数据集的剪枝和量化优化，但在SNN中同时进行剪枝和量化的研究还非常少。在这项工作中，我们提出了SpQuant-SNN，一种完全量化的脉冲神经网络，通过显著减少SNN上的操作实现了端到端的低精度。首先，我们为膜电位提出了一种仅整数的量化方案，采用堆叠替代梯度函数，这是一种简单而有效的方法，能够实现量化SNN训练的平滑学习过程。其次，我们利用膜电位先验实现空间通道剪枝，以降低SNN中层级的计算复杂度和浮点运算次数（FLOPs）。最后，为了进一步提高低精度和稀疏SNN的精度，我们为SNN训练提出了一种自适应可学习的电位阈值。SpQuant-SNN具有高度的生物适应性、最少的计算量和内存利用率，在基于事件的和静态图像数据集的多个SNN模型上实现了最先进的性能，包括图像分类和目标检测任务。与最先进的基线相比，所提出的SpQuant-SNN在分类和目标检测任务中实现了高达13倍的内存减少和>4.7倍的FLOPs减少，精度下降<1.8%。
