HCI Lab, College of Computer Science, Kookmin Univesity, 77 Jeongneung-ro, Souel 02707, Korea.
Electronics Engineering Department, Myongji University, 116 Myongji-ro, Cheoin-gu, Yongin-si 17058, Korea.
Sensors (Basel). 2021 Nov 8;21(21):7407. doi: 10.3390/s21217407.
The integral imaging system has received considerable research attention because it can be applied to real-time three-dimensional image displays with a continuous view angle without supplementary devices. Most previous approaches place a physical micro-lens array in front of the image, where each lens looks different depending on the viewing angle. A computational integral imaging system with a virtual micro-lens arrays has been proposed in order to provide flexibility for users to change micro-lens arrays and focal length while reducing distortions due to physical mismatches with the lens arrays. However, computational integral imaging methods only represent part of the whole image because the size of virtual lens arrays is much smaller than the given large-scale images when dealing with large-scale images. As a result, the previous approaches produce sub-aperture images with a small field of view and need additional devices for depth information to apply to integral imaging pickup systems. In this paper, we present a single image-based computational RGB-D integral imaging pickup system for a large field of view in real time. The proposed system comprises three steps: deep learning-based automatic depth map estimation from an RGB input image without the help of an additional device, a hierarchical integral imaging system for a large field of view in real time, and post-processing for optimized visualization of the failed pickup area using an inpainting method. Quantitative and qualitative experimental results verify the proposed approach's robustness.
积分成像系统因其可应用于无需附加设备的实时、连续视角的三维图像显示而受到广泛关注。大多数先前的方法在图像前放置物理微透镜阵列，每个透镜的视角不同。为了使用户能够在改变微透镜阵列和焦距的同时减少与透镜阵列物理不匹配引起的失真，提出了具有虚拟微透镜阵列的计算积分成像系统。然而，计算积分成像方法仅代表整个图像的一部分，因为在处理大尺寸图像时，虚拟透镜阵列的尺寸远小于给定的大尺寸图像。因此，以前的方法产生具有小视场的子孔径图像，并且需要额外的设备来获取深度信息，以应用于积分成像采集系统。在本文中，我们提出了一种基于单幅图像的实时大视场计算 RGB-D 积分成像采集系统。该系统包括三个步骤：从没有附加设备帮助的 RGB 输入图像中基于深度学习的自动深度图估计、用于实时大视场的分层积分成像系统以及使用图像修复方法进行优化的失败采集区域可视化的后处理。定量和定性实验结果验证了所提出方法的鲁棒性。
