Shen Xin, Markman Adam, Javidi Bahram
Appl Opt. 2017 Mar 20;56(9):D151-D157. doi: 10.1364/AO.56.00D151.
We present a method for three-dimensional (3D) profilometric reconstruction using flexible sensing integral imaging with object recognition and automatic occlusion removal. Two-dimensional images, known as elemental images (EIs), of a scene containing an object behind occlusion are captured by flexible sensing integral imaging using a moving camera randomly placed on a non-planar surface with unknown camera position and orientation. After 3D image acquisition, the unknown camera poses are estimated using the EIs and 3D reconstruction is performed based on flexible sensing integral imaging. Object recognition using the 3D reconstructed images is conducted to detect the object behind occlusion and estimate the object depth and position. Occlusion removal is then performed on the 2D EIs for the occluded object by computing variance maps of the scene. For each EI, occluded object pixels with low variance are replaced by object pixels from other perspectives using multi-view geometry. The new set of elemental images may be used to visualize the 3D profile of the scene containing the object without occlusion. Experiments are performed to validate the feasibility of the proposed method. To the best of our knowledge, this is the first report of applying flexible sensing integral imaging to profilometric reconstruction with object recognition and occlusion removal.
我们提出了一种三维（3D）轮廓测量重建方法，该方法采用具有目标识别和自动遮挡去除功能的柔性传感积分成像技术。使用随机放置在未知相机位置和方向的非平面表面上的移动相机，通过柔性传感积分成像捕获包含遮挡物后方物体的场景的二维图像，即元素图像（EI）。在获取3D图像后，使用EI估计未知相机姿态，并基于柔性传感积分成像进行3D重建。利用3D重建图像进行目标识别，以检测遮挡物后方的物体，并估计物体的深度和位置。然后，通过计算场景的方差图，对遮挡物体的二维EI进行遮挡去除。对于每个EI，使用多视图几何将方差较低的遮挡物体像素替换为其他视角的物体像素。新的元素图像集可用于可视化包含无遮挡物体的场景的3D轮廓。进行实验以验证所提方法的可行性。据我们所知，这是将柔性传感积分成像应用于具有目标识别和遮挡去除功能的轮廓测量重建的首次报道。
