IEEE Trans Neural Netw Learn Syst. 2022 Nov;33(11):6249-6262. doi: 10.1109/TNNLS.2021.3073016. Epub 2022 Oct 27.
Spiking neural networks (SNNs) based on the leaky integrate and fire (LIF) model have been applied to energy-efficient temporal and spatiotemporal processing tasks. Due to the bioplausible neuronal dynamics and simplicity, LIF-SNN benefits from event-driven processing, however, usually face the embarrassment of reduced performance. This may because, in LIF-SNN, the neurons transmit information via spikes. To address this issue, in this work, we propose a leaky integrate and analog fire (LIAF) neuron model so that analog values can be transmitted among neurons, and a deep network termed LIAF-Net is built on it for efficient spatiotemporal processing. In the temporal domain, LIAF follows the traditional LIF dynamics to maintain its temporal processing capability. In the spatial domain, LIAF is able to integrate spatial information through convolutional integration or fully connected integration. As a spatiotemporal layer, LIAF can also be used with traditional artificial neural network (ANN) layers jointly. In addition, the built network can be trained with backpropagation through time (BPTT) directly, which avoids the performance loss caused by ANN to SNN conversion. Experiment results indicate that LIAF-Net achieves comparable performance to the gated recurrent unit (GRU) and long short-term memory (LSTM) on bAbI question answering (QA) tasks and achieves state-of-the-art performance on spatiotemporal dynamic vision sensor (DVS) data sets, including MNIST-DVS, CIFAR10-DVS, and DVS128 Gesture, with much less number of synaptic weights and computational overhead compared with traditional networks built by LSTM, GRU, convolutional LSTM (ConvLSTM), or 3-D convolution (Conv3D). Compared with traditional LIF-SNN, LIAF-Net also shows dramatic accuracy gain on all these experiments. In conclusion, LIAF-Net provides a framework combining the advantages of both ANNs and SNNs for lightweight and efficient spatiotemporal information processing.
基于漏电流积分和放电（LIF）模型的尖峰神经网络（SNN）已应用于节能的时间和时空处理任务。由于具有生物逼真的神经元动力学和简单性，LIF-SNN 受益于事件驱动处理，但通常面临性能降低的尴尬。这可能是因为，在 LIF-SNN 中，神经元通过尖峰传递信息。为了解决这个问题，在这项工作中，我们提出了一种漏电流积分和模拟放电（LIAF）神经元模型，以便神经元之间可以传输模拟值，并在此基础上构建了一个名为 LIAF-Net 的深度网络，用于高效的时空处理。在时域中，LIAF 遵循传统的 LIF 动力学以保持其时间处理能力。在空域中，LIAF 能够通过卷积积分或全连接积分来整合空间信息。作为时空层，LIAF 也可以与传统的人工神经网络（ANN）层联合使用。此外，所构建的网络可以直接通过时间反向传播（BPTT）进行训练，这避免了 ANN 到 SNN 转换引起的性能损失。实验结果表明，LIAF-Net 在 bAbI 问答（QA）任务上的性能可与门控循环单元（GRU）和长短期记忆（LSTM）相媲美，在时空动态视觉传感器（DVS）数据集上也达到了最新水平，包括 MNIST-DVS、CIFAR10-DVS 和 DVS128 手势，与由 LSTM、GRU、卷积 LSTM（ConvLSTM）或 3-D 卷积（Conv3D）构建的传统网络相比，突触权重和计算开销要少得多。与传统的 LIF-SNN 相比，LIAF-Net 在所有这些实验中也显示出了显著的准确性提高。总之，LIAF-Net 为轻量级和高效的时空信息处理提供了一个结合了 ANN 和 SNN 优势的框架。
