Fan Xiaoting, Sun Long, Zhang Zhong, Liu Shuang, Durrani Tariq S
Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin 300387, China.
School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China.
Sensors (Basel). 2023 Aug 29;23(17):7488. doi: 10.3390/s23177488.
As an important representation of scenes in virtual reality and augmented reality, image stitching aims to generate a panoramic image with a natural field-of-view by stitching multiple images together, which are captured by different visual sensors. Existing deep-learning-based methods for image stitching only conduct a single deep homography to perform image alignment, which may produce inevitable alignment distortions. To address this issue, we propose a content-seam-preserving multi-alignment network (CSPM-Net) for visual-sensor-based image stitching, which could preserve the image content consistency and avoid seam distortions simultaneously. Firstly, a content-preserving deep homography estimation was designed to pre-align the input image pairs and reduce the content inconsistency. Secondly, an edge-assisted mesh warping was conducted to further align the image pairs, where the edge information is introduced to eliminate seam artifacts. Finally, in order to predict the final stitched image accurately, a content consistency loss was designed to preserve the geometric structure of overlapping regions between image pairs, and a seam smoothness loss is proposed to eliminate the edge distortions of image boundaries. Experimental results demonstrated that the proposed image-stitching method can provide favorable stitching results for visual-sensor-based images and outperform other state-of-the-art methods.
作为虚拟现实和增强现实中场景的重要表现形式，图像拼接旨在通过将多个由不同视觉传感器捕获的图像拼接在一起，生成具有自然视野的全景图像。现有的基于深度学习的图像拼接方法仅进行单次深度单应性变换来执行图像对齐，这可能会产生不可避免的对齐失真。为了解决这个问题，我们提出了一种用于基于视觉传感器的图像拼接的内容保留多对齐网络（CSPM-Net），它可以同时保持图像内容的一致性并避免接缝失真。首先，设计了一种内容保留深度单应性估计，对输入图像对进行预对齐并减少内容不一致性。其次，进行边缘辅助网格变形以进一步对齐图像对，其中引入边缘信息以消除接缝伪影。最后，为了准确预测最终的拼接图像，设计了一种内容一致性损失来保留图像对之间重叠区域的几何结构，并提出了一种接缝平滑损失来消除图像边界的边缘失真。实验结果表明，所提出的图像拼接方法可以为基于视觉传感器的图像提供良好的拼接结果，并且优于其他现有最先进的方法。
