Huang Zheng, Shi Wanxin, Wu Shukai, Wang Yaode, Yang Sigang, Chen Hongwei
Department of Electronic Engineering, Tsinghua University, Beijing 100084, China.
Beijing National Research Center for Information Science and Technology, Beijing 100084, China.
Sci Adv. 2024 Jul 26;10(30):eado8516. doi: 10.1126/sciadv.ado8516.
Moving computation units closer to sensors is becoming a promising approach to addressing bottlenecks in computing speed, power consumption, and data storage. Pre-sensor computing with optical neural networks (ONNs) allows extensive processing. However, the lack of nonlinear activation and dependence on laser input limits the computational capacity, practicality, and scalability. A compact and passive multilayer ONN (MONN) is proposed, which has two convolution layers and an inserted nonlinear layer, performing pre-sensor computations with designed passive masks and a quantum dot film for incoherent light. MONN has an optical length as short as 5 millimeters, two orders of magnitude smaller than state-of-the-art lens-based ONNs. MONN outperforms linear single-layer ONN across various vision tasks, off-loading up to 95% of computationally expensive operations into optics from electronics. Motivated by MONN, a paradigm is emerging for mobile vision, fulfilling the demands for practicality, miniaturization, and low power consumption.
将计算单元移近传感器正成为解决计算速度、功耗和数据存储瓶颈的一种有前途的方法。利用光学神经网络（ONN）进行传感器前计算可实现广泛处理。然而，缺乏非线性激活以及对激光输入的依赖限制了计算能力、实用性和可扩展性。本文提出了一种紧凑的无源多层光学神经网络（MONN），它有两个卷积层和一个插入的非线性层，通过设计的无源掩膜和用于非相干光的量子点薄膜进行传感器前计算。MONN的光学长度短至5毫米，比基于透镜的最先进ONN小两个数量级。在各种视觉任务中，MONN的性能优于线性单层ONN，可将高达95%的计算昂贵操作从电子设备卸载到光学器件中。受MONN的启发，一种适用于移动视觉的范式正在兴起，满足了实用性、小型化和低功耗的需求。
