Park Keon-Woo, Shim Yoo-Jeong, Lee Myeong-Jin
The Information Technology & Mobile Communications Biz., Samsung Electronics, Suwon-si 16677, Gyeonggi-do, Korea.
School of Electronics and Information Engineering, Korea Aerospace University, Goyang-si 10540, Gyeonggi-do, Korea.
Sensors (Basel). 2021 Jun 10;21(12):4020. doi: 10.3390/s21124020.
In this paper, we propose a semantic segmentation-based static video stitching method to reduce parallax and misalignment distortion for sports stadium scenes with dynamic foreground objects. First, video frame pairs for stitching are divided into segments of different classes through semantic segmentation. Region-based stitching is performed on matched segment pairs, assuming that segments of the same semantic class are on the same plane. Second, to prevent degradation of the stitching quality of plain or noisy videos, the homography for each matched segment pair is estimated using the temporally consistent feature points. Finally, the stitched video frame is synthesized by stacking the stitched matched segment pairs and the foreground segments to the reference frame plane by descending order of the area. The performance of the proposed method is evaluated by comparing the subjective quality, geometric distortion, and pixel distortion of video sequences stitched using the proposed and conventional methods. The proposed method is shown to reduce parallax and misalignment distortion in segments with plain texture or large parallax, and significantly improve geometric distortion and pixel distortion compared to conventional methods.
在本文中，我们提出了一种基于语义分割的静态视频拼接方法，以减少具有动态前景物体的体育场馆场景中的视差和错位失真。首先，通过语义分割将用于拼接的视频帧对划分为不同类别的片段。对匹配的片段对执行基于区域的拼接，假设相同语义类别的片段位于同一平面上。其次，为了防止纯色或有噪视频的拼接质量下降，使用时间上一致的特征点估计每个匹配片段对的单应性。最后，通过将拼接后的匹配片段对和前景片段按面积降序堆叠到参考帧平面上来合成拼接后的视频帧。通过比较使用所提出的方法和传统方法拼接的视频序列的主观质量、几何失真和像素失真来评估所提出方法的性能。结果表明，与传统方法相比，所提出的方法能够减少纯色纹理或大视差片段中的视差和错位失真，并显著改善几何失真和像素失真。
