Li Yiyang, Xiao T Patrick, Bennett Christopher H, Isele Erik, Melianas Armantas, Tao Hanbo, Marinella Matthew J, Salleo Alberto, Fuller Elliot J, Talin A Alec
Sandia National Laboratories, Livermore, CA, United States.
Sandia National Laboratories, Albuquerque, NM, United States.
Front Neurosci. 2021 Apr 8;15:636127. doi: 10.3389/fnins.2021.636127. eCollection 2021.
In-memory computing based on non-volatile resistive memory can significantly improve the energy efficiency of artificial neural networks. However, accurate training has been challenging due to the nonlinear and stochastic switching of the resistive memory elements. One promising analog memory is the electrochemical random-access memory (ECRAM), also known as the redox transistor. Its low write currents and linear switching properties across hundreds of analog states enable accurate and massively parallel updates of a full crossbar array, which yield rapid and energy-efficient training. While simulations predict that ECRAM based neural networks achieve high training accuracy at significantly higher energy efficiency than digital implementations, these predictions have not been experimentally achieved. In this work, we train a 3 × 3 array of ECRAM devices that learns to discriminate several elementary logic gates (AND, OR, NAND). We record the evolution of the network's synaptic weights during parallel (on-line) training, with outer product updates. Due to linear and reproducible device switching characteristics, our crossbar simulations not only accurately simulate the epochs to convergence, but also quantitatively capture the evolution of weights in individual devices. The implementation of the first parallel training together with strong agreement with simulation results provides a significant advance toward developing ECRAM into larger crossbar arrays for artificial neural network accelerators, which could enable orders of magnitude improvements in energy efficiency of deep neural networks.
基于非易失性电阻式存储器的内存计算能够显著提高人工神经网络的能源效率。然而,由于电阻式存储元件的非线性和随机开关特性,精确训练一直颇具挑战。一种很有前景的模拟存储器是电化学随机存取存储器(ECRAM),也称为氧化还原晶体管。其低写入电流以及在数百个模拟状态下的线性开关特性,能够对完整的交叉阵列进行精确且大规模并行更新,从而实现快速且节能的训练。虽然模拟结果预测,基于ECRAM的神经网络在能效方面比数字实现方式显著更高的情况下仍能达到较高的训练精度,但这些预测尚未通过实验得到验证。在这项工作中,我们训练了一个由3×3个ECRAM器件组成的阵列,使其学会区分几种基本逻辑门(与门、或门、与非门)。我们在并行(在线)训练过程中,通过外积更新记录网络突触权重的演变。由于器件具有线性且可重复的开关特性,我们的交叉阵列模拟不仅能准确模拟达到收敛所需的轮次,还能定量捕捉单个器件中权重的演变。首次并行训练的实现以及与模拟结果的高度吻合,为将ECRAM发展成用于人工神经网络加速器的更大交叉阵列迈出了重要一步,这有望使深度神经网络的能源效率实现数量级的提升。