Liu Shu-Bin, Liu Xu-Ning, Fan Wei-Jie, Zhang Meng-Xuan, Li Lei
School of Electronics and Information Engineering, Sichuan University, Chengdu 610065, China.
Faculty of Science, The University of Melbourne, Victoria, 3010, Australia.
Nanophotonics. 2024 Apr 23;13(16):2879-2890. doi: 10.1515/nanoph-2023-0782. eCollection 2024 Jul.
The growing interests have been witnessed in the evolution and improvement of artificial compound eyes (CE) inspired by arthropods. However, the existing CE cameras are suffering from a defocusing problem due to the incompatibility with commercial CMOS cameras. Inspired by the CEs of South American Shrimps, we report a compact biologically inspired camera that enables wide-field-of-view (FOV), high-resolution imaging and sensitive 3D moving trajectory reconstruction. To overcome the defocusing problem, a deep learning architecture with distance regulation is proposed to achieve wide-range-clear imaging, without any hardware or complex front-end design, which greatly reduces system complexity and size. The architecture is composed of a variant of Unet and Pyramid-multi-scale attention, with designed short, middle and long distance regulation. Compared to the current competitive well-known models, our method is at least 2 dB ahead. Here we describe the high-resolution computational-CE camera with 271 ommatidia, with a weight of 5.4 g an area of 3 × 3 cm and 5-mm thickness, which achieves compatibility and integration of CE with commercial CMOS. The experimental result illustrates this computational-CE camera has competitive advantages in enhanced resolution and sensitive 3D live moving trajectory reconstruction. The compact camera has promising applications in nano-optics fields such as medical endoscopy, panoramic imaging and vision robotics.
受节肢动物启发，人工复眼（CE）的进化和改进引发了越来越多的关注。然而，由于与商用CMOS相机不兼容，现有的CE相机存在散焦问题。受南美虾复眼的启发，我们报道了一种紧凑的仿生相机，它能够实现宽视场（FOV）、高分辨率成像以及灵敏的三维运动轨迹重建。为了克服散焦问题，我们提出了一种带有距离调节功能的深度学习架构，无需任何硬件或复杂的前端设计即可实现大范围清晰成像，这大大降低了系统复杂度和尺寸。该架构由Unet变体和金字塔多尺度注意力组成，并设计了短、中、长距离调节。与当前具有竞争力的知名模型相比，我们的方法至少领先2 dB。在此，我们描述了一种具有271个小眼的高分辨率计算型CE相机，其重量为5.4 g，面积为3×3 cm，厚度为5 mm，实现了CE与商用CMOS的兼容与集成。实验结果表明，这种计算型CE相机在提高分辨率和灵敏的三维活体运动轨迹重建方面具有竞争优势。这种紧凑的相机在医学内窥镜检查、全景成像和视觉机器人等纳米光学领域具有广阔的应用前景。
