Cui Kaiyu, Rao Shijie, Xu Sheng, Huang Yidong, Cai Xusheng, Huang Zhilei, Wang Yu, Feng Xue, Liu Fang, Zhang Wei, Li Yali, Wang Shengjin
Department of Electronic Engineering, Tsinghua University, Beijing, China.
Beijing Seetrum Technology Co., Beijing, China.
Nat Commun. 2025 Jan 2;16(1):81. doi: 10.1038/s41467-024-55558-3.
Optical neural networks are considered next-generation physical implementations of artificial neural networks, but their capabilities are limited by on-chip integration scale and requirement for coherent light sources. This study proposes a spectral convolutional neural network (SCNN) with matter meta-imaging. The optical convolutional layer is implemented by integrating very large-scale and pixel-aligned spectral filters on CMOS image sensor. It facilitates highly parallel spectral vector-inner products of incident incoherent natural light i.e., the direct information carrier, which empowers in-sensor optical analog computing at extremely high energy efficiency. To the best of our knowledge, this is the first integrated optical computing utilizing natural light. We employ the same SCNN chip for completely different real-world complex tasks and achieve accuracies of over 96% for pathological diagnosis and almost 100% for face anti-spoofing at video rates. These results indicate a feasible and scalable in-sensor edge computing chip of natural light for various portable terminals.
光学神经网络被认为是人工神经网络的下一代物理实现方式，但其能力受到片上集成规模和对相干光源的要求的限制。本研究提出了一种具有物质元成像的光谱卷积神经网络（SCNN）。光学卷积层通过在CMOS图像传感器上集成超大规模且像素对齐的光谱滤波器来实现。它便于对入射的非相干自然光（即直接信息载体）进行高度并行的光谱向量内积运算，从而以极高的能量效率实现传感器内的光学模拟计算。据我们所知，这是首次利用自然光的集成光学计算。我们将同一个SCNN芯片用于完全不同的现实世界复杂任务，在视频帧率下，病理诊断的准确率超过96%，面部反欺骗的准确率几乎达到100%。这些结果表明，一种适用于各种便携式终端的、可行且可扩展的自然光传感器内边缘计算芯片是可行的。
