Lin Haipeng, Ou Jiali, Fan Zhen, Yan Xiaobing, Hu Wenjie, Cui Boyuan, Xu Jikang, Li Wenjie, Chen Zhiwei, Yang Biao, Liu Kun, Mo Linyuan, Li Meixia, Lu Xubing, Zhou Guofu, Gao Xingsen, Liu Jun-Ming
Institute for Advanced Materials and Guangdong Provincial Key Laboratory of Optical Information Materials and Technology, South China Academy of Advanced Optoelectronics, South China Normal University, Guangzhou, China.
Key Laboratory of Brain-Like Neuromorphic Devices and Systems of Hebei Province, Hebei Key Laboratory of Photo-Electricity Information and Materials, Hebei University, Baoding, China.
Nat Commun. 2025 Jan 7;16(1):421. doi: 10.1038/s41467-024-55508-z.
In-sensor computing has emerged as an ultrafast and low-power technique for next-generation machine vision. However, in situ training of in-sensor computing systems remains challenging due to the demands for both high-performance devices and efficient programming schemes. Here, we experimentally demonstrate the in situ training of an in-sensor artificial neural network (ANN) based on ferroelectric photosensors (FE-PSs). Our FE-PS exhibits self-powered, fast (<30 μs), and multilevel (>4 bits) photoresponses, as well as long retention (50 days), high endurance (10), high write speed (100 ns), and small cycle-to-cycle and device-to-device variations (~0.66% and ~2.72%, respectively), all of which are desirable for the in situ training. Additionally, a bi-directional closed-loop programming scheme is developed, achieving a precise and efficient weight update for the FE-PS. Using this programming scheme, an in-sensor ANN based on the FE-PSs is trained in situ to recognize traffic signs for commanding a prototype autonomous vehicle. Moreover, this in-sensor ANN operates 50 times faster than a von Neumann machine vision system. This study paves the way for the development of in-sensor computing systems with in situ training capability, which may find applications in new data-streaming machine vision tasks.
传感器内计算已成为一种用于下一代机器视觉的超快速、低功耗技术。然而，由于对高性能器件和高效编程方案的需求，传感器内计算系统的原位训练仍然具有挑战性。在此，我们通过实验证明了基于铁电光电传感器（FE-PS）的传感器内人工神经网络（ANN）的原位训练。我们的FE-PS表现出自供电、快速（<30微秒）和多级（>4位）光响应，以及长保持时间（50天）、高耐久性（10次）、高写入速度（100纳秒）和小的周期到周期以及器件到器件变化（分别约为0.66%和约2.72%），所有这些都是原位训练所期望的。此外，还开发了一种双向闭环编程方案，实现了FE-PS的精确高效权重更新。使用该编程方案，基于FE-PS的传感器内ANN被原位训练以识别交通标志，用于指挥原型自动驾驶车辆。此外，该传感器内ANN的运行速度比冯·诺依曼机器视觉系统快50倍。这项研究为具有原位训练能力的传感器内计算系统的发展铺平了道路，该系统可能在新的数据流机器视觉任务中找到应用。
