Noh Kyungmi, Kwak Hyunjeong, Son Jeonghoon, Kim Seungkun, Um Minseong, Kang Minil, Kim Doyoon, Ji Wonjae, Lee Junyong, Jo HwiJeong, Woo Jiyong, Lee Hyung-Min, Kim Seyoung
Department of Materials Science and Engineering, Pohang University of Science and Technology, Pohang 37673, Republic of Korea.
School of Electrical Engineering, Korea University, Seoul 02841, Republic of Korea.
Sci Adv. 2024 Jun 14;10(24):eadl3350. doi: 10.1126/sciadv.adl3350.
We present the fabrication of 4 K-scale electrochemical random-access memory (ECRAM) cross-point arrays for analog neural network training accelerator and an electrical characteristic of an 8 × 8 ECRAM array with a 100% yield, showing excellent switching characteristics, low cycle-to-cycle, and device-to-device variations. Leveraging the advances of the ECRAM array, we showcase its efficacy in neural network training using the Tiki-Taka version 2 algorithm (TTv2) tailored for non-ideal analog memory devices. Through an experimental study using ECRAM devices, we investigate the influence of retention characteristics on the training performance of TTv2, revealing that the relative location of the retention convergence point critically determines the available weight range and, consequently, affects the training accuracy. We propose a retention-aware zero-shifting technique designed to optimize neural network training performance, particularly in scenarios involving cross-point devices with limited retention times. This technique ensures robust and efficient analog neural network training despite the practical constraints posed by analog cross-point devices.
我们展示了用于模拟神经网络训练加速器的4K规模电化学随机存取存储器（ECRAM）交叉点阵列的制造，以及一个良率为100%的8×8 ECRAM阵列的电学特性，该阵列显示出优异的开关特性、低周期到周期以及器件到器件的变化。利用ECRAM阵列的进展，我们展示了其在使用针对非理想模拟存储器件定制的Tiki-Taka版本2算法（TTv2）进行神经网络训练中的功效。通过使用ECRAM器件的实验研究，我们研究了保持特性对TTv2训练性能的影响，揭示了保持收敛点的相对位置关键地决定了可用权重范围，进而影响训练精度。我们提出了一种保持感知零移位技术，旨在优化神经网络训练性能，特别是在涉及保持时间有限的交叉点器件的场景中。尽管模拟交叉点器件带来了实际限制，但该技术确保了强大而高效的模拟神经网络训练。
