Wang Shaocong, Gao Yizhao, Li Yi, Zhang Woyu, Yu Yifei, Wang Bo, Lin Ning, Chen Hegan, Zhang Yue, Jiang Yang, Wang Dingchen, Chen Jia, Dai Peng, Jiang Hao, Lin Peng, Zhang Xumeng, Qi Xiaojuan, Xu Xiaoxin, So Hayden, Wang Zhongrui, Shang Dashan, Liu Qi, Cheng Kwang-Ting, Liu Ming
Department of Electrical and Electronic Engineering, The University of Hong Kong, Hong Kong, China.
Key Lab of Fabrication Technologies for Integrated Circuits Institute of Microelectronics, Chinese Academy of Sciences, 100029, Beijing, China.
Nat Commun. 2025 Jan 23;16(1):960. doi: 10.1038/s41467-025-56079-3.
Visual sensors, including 3D light detection and ranging, neuromorphic dynamic vision sensor, and conventional frame cameras, are increasingly integrated into edge-side intelligent machines. However, their data are heterogeneous, causing complexity in system development. Moreover, conventional digital hardware is constrained by von Neumann bottleneck and the physical limit of transistor scaling. The computational demands of training ever-growing models further exacerbate these challenges. We propose a hardware-software co-designed random resistive memory-based deep extreme point learning machine. Data-wise, the multi-sensory data are unified as point set and processed universally. Software-wise, most weights are exempted from training. Hardware-wise, nanoscale resistive memory enables collocation of memory and processing, and leverages the inherent programming stochasticity for generating random weights. The co-design system is validated on 3D segmentation (ShapeNet), event recognition (DVS128 Gesture), and image classification (Fashion-MNIST) tasks, achieving accuracy comparable to conventional systems while delivering 6.78 × /21.04 × /15.79 × energy efficiency improvements and 70.12%/89.46%/85.61% training cost reductions.
视觉传感器，包括三维激光雷达、神经形态动态视觉传感器和传统的帧相机，正越来越多地集成到边缘智能机器中。然而，它们的数据是异构的，这给系统开发带来了复杂性。此外，传统数字硬件受到冯·诺依曼瓶颈和晶体管缩放物理极限的限制。训练不断增长的模型的计算需求进一步加剧了这些挑战。我们提出了一种基于硬件-软件协同设计的随机电阻式存储器的深度极限学习机。在数据方面，多感官数据被统一为点集并进行统一处理。在软件方面，大多数权重无需训练。在硬件方面，纳米级电阻式存储器实现了存储与处理的搭配，并利用固有的编程随机性来生成随机权重。该协同设计系统在三维分割（ShapeNet）、事件识别（DVS128手势）和图像分类（Fashion-MNIST）任务上得到了验证，在实现与传统系统相当的准确率的同时，能效提高了6.78倍/21.04倍/15.79倍，训练成本降低了70.12%/89.46%/85.61%。
