IEEE Trans Neural Netw Learn Syst. 2023 Sep;34(9):5841-5855. doi: 10.1109/TNNLS.2021.3131356. Epub 2023 Sep 1.
Spiking neural networks (SNNs), inspired by the neuronal network in the brain, provide biologically relevant and low-power consuming models for information processing. Existing studies either mimic the learning mechanism of brain neural networks as closely as possible, for example, the temporally local learning rule of spike-timing-dependent plasticity (STDP), or apply the gradient descent rule to optimize a multilayer SNN with fixed structure. However, the learning rule used in the former is local and how the real brain might do the global-scale credit assignment is still not clear, which means that those shallow SNNs are robust but deep SNNs are difficult to be trained globally and could not work so well. For the latter, the nondifferentiable problem caused by the discrete spike trains leads to inaccuracy in gradient computing and difficulties in effective deep SNNs. Hence, a hybrid solution is interesting to combine shallow SNNs with an appropriate machine learning (ML) technique not requiring the gradient computing, which is able to provide both energy-saving and high-performance advantages. In this article, we propose a HybridSNN, a deep and strong SNN composed of multiple simple SNNs, in which data-driven greedy optimization is used to build powerful classifiers, avoiding the derivative problem in gradient descent. During the training process, the output features (spikes) of selected weak classifiers are fed back to the pool for the subsequent weak SNN training and selection. This guarantees HybridSNN not only represents the linear combination of simple SNNs, as what regular AdaBoost algorithm generates, but also contains neuron connection information, thus closely resembling the neural networks of a brain. HybridSNN has the benefits of both low power consumption in weak units and overall data-driven optimizing strength. The network structure in HybridSNN is learned from training samples, which is more flexible and effective compared with existing fixed multilayer SNNs. Moreover, the topological tree of HybridSNN resembles the neural system in the brain, where pyramidal neurons receive thousands of synaptic input signals through their dendrites. Experimental results show that the proposed HybridSNN is highly competitive among the state-of-the-art SNNs.
尖峰神经网络（SNN）受大脑神经网络启发，为信息处理提供具有生物学意义且低功耗的模型。现有研究要么尽可能模仿大脑神经网络的学习机制，例如尖峰时间依赖可塑性（STDP）的暂态局部学习规则，要么应用梯度下降规则优化具有固定结构的多层 SNN。然而，前者使用的学习规则是局部的，而真正的大脑如何进行全局规模的信用分配还不清楚，这意味着那些浅层 SNN 具有鲁棒性，但深层 SNN 很难全局训练，效果也不佳。对于后者，由于离散尖峰序列引起的不可微问题导致梯度计算不准确，并且难以有效地进行深层 SNN。因此，将浅层 SNN 与不需要梯度计算的适当机器学习（ML）技术相结合的混合解决方案很有趣，它能够提供节能和高性能优势。在本文中，我们提出了一种 HybridSNN，这是一种由多个简单 SNN 组成的深度且强大的 SNN，其中使用数据驱动的贪婪优化来构建强大的分类器，从而避免了梯度下降中的导数问题。在训练过程中，选择的弱分类器的输出特征（尖峰）会被反馈到池，以用于后续的弱 SNN 训练和选择。这保证了 HybridSNN 不仅表示简单 SNN 的线性组合，就像常规 AdaBoost 算法生成的那样，而且还包含神经元连接信息，因此与大脑神经网络非常相似。HybridSNN 具有弱单元低功耗和整体数据驱动优化强度的优势。HybridSNN 的网络结构是从训练样本中学习的，与现有的固定多层 SNN 相比，它更加灵活和有效。此外，HybridSNN 的拓扑树类似于大脑中的神经系统，其中金字塔神经元通过树突接收数千个突触输入信号。实验结果表明，所提出的 HybridSNN 在最先进的 SNN 中具有很强的竞争力。
