Lin Shijie, Zheng Guangze, Wang Ziwei, Han Ruihua, Xing Wanli, Zhang Zeqing, Peng Yifan, Pan Jia
Department of Computer Science, The University of Hong Kong, Pokfulam Rd, Hong Kong SAR, China.
College of Engineering and Computer Science Australian National University, Canberra, ACT, Australia.
Nat Commun. 2024 Dec 30;15(1):10781. doi: 10.1038/s41467-024-54789-8.
Proper exposure settings are crucial for modern machine vision cameras to accurately convert light into clear images. However, traditional auto-exposure solutions are vulnerable to illumination changes, splitting the continuous acquisition of unsaturated images, which significantly degrades the overall performance of underlying intelligent systems. Here we present the neuromorphic exposure control (NEC) system. This system effectively alleviates the longstanding saturation problem at its core by exploiting bio-principles found in peripheral vision to compute a trilinear event double integral (TEDI). This approach enables accurate connections between events and frames in the physics space for swift irradiance prediction, ultimately facilitating rapid control parameter updates. Our experimental results demonstrate the remarkable efficiency, low latency, superior generalization capability, and bio-inspired nature of the NEC in delivering timely and robust neuromorphic synergy for lighting-robust machine vision across a wide range of real-world applications. These applications encompass autonomous driving, mixed-reality, and three-dimensional reconstruction.
合适的曝光设置对于现代机器视觉相机准确地将光转换为清晰图像至关重要。然而，传统的自动曝光解决方案容易受到光照变化的影响，导致无法连续采集不饱和图像，这显著降低了底层智能系统的整体性能。在此，我们提出了神经形态曝光控制（NEC）系统。该系统通过利用周边视觉中发现的生物原理来计算三线性事件二重积分（TEDI），从核心上有效缓解了长期存在的饱和度问题。这种方法能够在物理空间中实现事件与帧之间的精确关联，以便快速进行辐照度预测，最终促进控制参数的快速更新。我们的实验结果表明，NEC在为广泛的实际应用提供及时且强大的神经形态协同以实现光照鲁棒的机器视觉方面，具有显著的效率、低延迟、卓越的泛化能力以及受生物启发的特性。这些应用包括自动驾驶、混合现实和三维重建。
