Liao Yan, Deng Ning, Wu Huaqiang, Gao Bin, Zhang Qingtian, Qian He
Institute of Microelectronics, Tsinghua University, Beijing, China.
Front Neurosci. 2018 Mar 16;12:167. doi: 10.3389/fnins.2018.00167. eCollection 2018.
The parallel updating scheme of RRAM-based analog neuromorphic systems based on sign stochastic gradient descent (SGD) can dramatically accelerate the training of neural networks. However, sign SGD can decrease accuracy. Also, some non-ideal factors of RRAM devices, such as intrinsic variations and the quantity of intermediate states, may significantly damage their convergence. In this paper, we analyzed the effects of these issues on the parallel updating scheme and found that it performed poorly on the task of MNIST recognition when the number of intermediate states was limited or the variation was too large. Thus, we propose a weighted synapse method to optimize the parallel updating scheme. Weighted synapses consist of major and minor synapses with different gain factors. Such a method can be widely used in RRAM-based analog neuromorphic systems to increase the number of equivalent intermediate states exponentially. The proposed method also generates a more suitable Δ , diminishing the distortion caused by sign SGD. Unlike when several RRAM cells are combined to achieve higher resolution, there are no carry operations for weighted synapses, even if a saturation on the minor synapses occurs. The proposed method also simplifies the circuit overhead, rendering it highly suitable to the parallel updating scheme. With the aid of weighted synapses, convergence is highly optimized, and the error rate decreases significantly. Weighted synapses are also robust against the intrinsic variations of RRAM devices.
基于符号随机梯度下降（SGD）的基于电阻式随机存取存储器（RRAM）的模拟神经形态系统的并行更新方案可以显著加速神经网络的训练。然而，符号SGD会降低准确率。此外，RRAM器件的一些非理想因素，如固有变化和中间状态数量，可能会严重破坏其收敛性。在本文中，我们分析了这些问题对并行更新方案的影响，发现当中间状态数量有限或变化过大时，该方案在MNIST识别任务上表现不佳。因此，我们提出了一种加权突触方法来优化并行更新方案。加权突触由具有不同增益因子的主突触和次突触组成。这种方法可以广泛应用于基于RRAM的模拟神经形态系统，以指数方式增加等效中间状态的数量。所提出的方法还会生成更合适的Δ，减少由符号SGD引起的失真。与将几个RRAM单元组合以实现更高分辨率的情况不同，加权突触不存在进位操作，即使次突触出现饱和。所提出的方法还简化了电路开销，使其非常适合并行更新方案。借助加权突触，收敛得到了高度优化，错误率显著降低。加权突触对RRAM器件的固有变化也具有鲁棒性。
