Dorzhigulov Anuar, Saxena Vishal
AMPIC Lab, Department of Electrical and Electronic Engineering, University of Delaware, Newark, DE, United States.
Front Neurosci. 2023 Jul 18;17:1177592. doi: 10.3389/fnins.2023.1177592. eCollection 2023.
We increasingly rely on deep learning algorithms to process colossal amount of unstructured visual data. Commonly, these deep learning algorithms are deployed as software models on digital hardware, predominantly in data centers. Intrinsic high energy consumption of Cloud-based deployment of deep neural networks (DNNs) inspired researchers to look for alternatives, resulting in a high interest in Spiking Neural Networks (SNNs) and dedicated mixed-signal neuromorphic hardware. As a result, there is an emerging challenge to transfer DNN architecture functionality to energy-efficient spiking non-volatile memory (NVM)-based hardware with minimal loss in the accuracy of visual data processing. Convolutional Neural Network (CNN) is the staple choice of DNN for visual data processing. However, the lack of analog-friendly spiking implementations and alternatives for some core CNN functions, such as MaxPool, hinders the conversion of CNNs into the spike domain, thus hampering neuromorphic hardware development. To address this gap, in this work, we propose MaxPool with temporal multiplexing for Spiking CNNs (SCNNs), which is amenable for implementation in mixed-signal circuits. In this work, we leverage the temporal dynamics of internal membrane potential of Integrate & Fire neurons to enable MaxPool decision-making in the spiking domain. The proposed MaxPool models are implemented and tested within the SCNN architecture using a modified version of the aihwkit framework, a PyTorch-based toolkit for modeling and simulating hardware-based neural networks. The proposed spiking MaxPool scheme can decide even before the complete spatiotemporal input is applied, thus selectively trading off latency with accuracy. It is observed that by allocating just 10% of the spatiotemporal input window for a pooling decision, the proposed spiking MaxPool achieves up to 61.74% accuracy with a 2-bit weight resolution in the CIFAR10 dataset classification task after training with back propagation, with only about 1% performance drop compared to 62.78% accuracy of the 100% spatiotemporal window case with the 2-bit weight resolution to reflect foundry-integrated ReRAM limitations. In addition, we propose the realization of one of the proposed spiking MaxPool techniques in an NVM crossbar array along with periphery circuits designed in a 130nm CMOS technology. The energy-efficiency estimation results show competitive performance compared to recent neuromorphic chip designs.
我们越来越依赖深度学习算法来处理海量的非结构化视觉数据。通常，这些深度学习算法作为软件模型部署在数字硬件上，主要是在数据中心。基于云的深度神经网络（DNN）部署存在固有的高能耗问题，这促使研究人员寻找替代方案，从而引发了对脉冲神经网络（SNN）和专用混合信号神经形态硬件的高度关注。因此，出现了一个新的挑战，即如何将DNN架构功能转移到基于节能脉冲非易失性存储器（NVM）的硬件上，同时在视觉数据处理精度上的损失最小。卷积神经网络（CNN）是用于视觉数据处理的DNN的主要选择。然而，缺乏对模拟友好的脉冲实现方式以及一些核心CNN功能（如最大池化（MaxPool））的替代方案，阻碍了将CNN转换到脉冲域，从而妨碍了神经形态硬件的发展。为了弥补这一差距，在这项工作中，我们提出了用于脉冲卷积神经网络（SCNN）的具有时间复用的MaxPool，它适合在混合信号电路中实现。在这项工作中，我们利用积分发放神经元内部膜电位的时间动态特性，在脉冲域中实现MaxPool决策。所提出的MaxPool模型在SCNN架构内使用aihwkit框架的修改版本进行实现和测试，aihwkit是一个基于PyTorch的用于建模和模拟基于硬件的神经网络的工具包。所提出的脉冲MaxPool方案甚至可以在完整的时空输入应用之前就做出决策，从而在延迟和精度之间进行选择性权衡。据观察，通过仅为池化决策分配时空输入窗口的10%，所提出的脉冲MaxPool在使用反向传播训练后的CIFAR10数据集分类任务中，以2位权重分辨率实现了高达61.74%的准确率，与2位权重分辨率的100%时空窗口情况的62.78%准确率相比，性能仅下降约1%，以反映代工厂集成的电阻式随机存取存储器（ReRAM）的局限性。此外，我们提出在一个NVM交叉阵列中实现所提出的一种脉冲MaxPool技术，并结合采用130nm CMOS技术设计的外围电路。能效估计结果表明，与最近的神经形态芯片设计相比，具有竞争力。
