School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China.
Australian Artificial Intelligence Institute, Faculty of Engineering and Information Technology, University of Technology Sydney, Ultimo, NSW 2007, Australia.
Neural Netw. 2021 Dec;144:565-572. doi: 10.1016/j.neunet.2021.09.016. Epub 2021 Sep 27.
At present, in the new hardware design work of deep learning, memristor as a non-volatile memory with computing power has become a research hotspot. The weights in the deep neural network are the floating-point number. Writing a floating-point value into a memristor will result in a loss of accuracy, and the writing process will take more time. The binarized neural network (BNN) binarizes the weights and activation values that were originally floating-point numbers to +1 and -1. This will greatly reduce the storage space consumption and time consumption of programming the resistance value of the memristor. Furthermore, this will help to simplify the programming of memristors in deep neural network circuits and speed up the inference process. This paper provides a complete solution for implementing memristive BNN. Furthermore, we improved the design of the memristor crossbar by converting the input feature map and kernel before performing the convolution operation that can ensure the sign of the input voltage of each port constant. Therefore, we do not need to determine the sign of the input voltage required by the port in advance which simplifies the process of inputting the feature map elements to each port of the crossbar in the form of voltage. At the same time, in order to ensure that the output of the current convolution layer can be directly used as the input of the next layer, we have added a corresponding processing circuit, which integrates batch-normalization and binarization operations.
目前，在深度学习的新硬件设计工作中，具有计算能力的忆阻器作为非易失性存储器已成为研究热点。深度神经网络中的权重是浮点数。将浮点数写入忆阻器会导致精度损失，并且写入过程将花费更多时间。二进制神经网络 (BNN) 将原本是浮点数的权重和激活值二值化为+1 和-1。这将大大减少存储消耗和编程忆阻器电阻值的时间消耗。此外，这有助于简化深度神经网络电路中忆阻器的编程并加快推理过程。本文为实现忆阻 BNN 提供了完整的解决方案。此外，我们通过在执行卷积操作之前转换输入特征图和核，改进了忆阻器交叉开关的设计，这可以确保每个端口输入电压的符号保持不变。因此，我们不需要预先确定端口所需的输入电压的符号，这简化了以电压形式将特征图元素输入到交叉开关每个端口的过程。同时，为了确保当前卷积层的输出可以直接用作下一层的输入，我们添加了相应的处理电路，该电路集成了批量归一化和二值化操作。
