Ling Suiyi, Li Jing, Che Zhaohui, Min Xiongkuo, Zhai Guangtao, Le Callet Patrick
IEEE Trans Image Process. 2021;30:517-531. doi: 10.1109/TIP.2020.3037504. Epub 2020 Nov 24.
Virtual viewpoints synthesis is an essential process for many immersive applications including Free-viewpoint TV (FTV). A widely used technique for viewpoints synthesis is Depth-Image-Based-Rendering (DIBR) technique. However, such technique may introduce challenging non-uniform spatial-temporal structure-related distortions. Most of the existing state-of-the-art quality metrics fail to handle these distortions, especially the temporal structure inconsistencies observed during the switch of different viewpoints. To tackle this problem, an elastic metric and multi-scale trajectory based video quality metric (EM-VQM) is proposed in this paper. Dense motion trajectory is first used as a proxy for selecting temporal sensitive regions, where local geometric distortions might significantly diminish the perceived quality. Afterwards, the amount of temporal structure inconsistencies and unsmooth viewpoints transitions are quantified by calculating 1) the amount of motion trajectory deformations with elastic metric and, 2) the spatial-temporal structural dissimilarity. According to the comprehensive experimental results on two FTV video datasets, the proposed metric outperforms the state-of-the-art metrics designed for free-viewpoint videos significantly and achieves a gain of 12.86% and 16.75% in terms of median Pearson linear correlation coefficient values on the two datasets compared to the best one, respectively.
虚拟视点合成是包括自由视点电视（FTV）在内的许多沉浸式应用中的一个重要过程。一种广泛使用的视点合成技术是基于深度图像的渲染（DIBR）技术。然而，这种技术可能会引入具有挑战性的与非均匀时空结构相关的失真。现有的大多数先进质量指标都无法处理这些失真，尤其是在不同视点切换期间观察到的时间结构不一致性。为了解决这个问题，本文提出了一种基于弹性度量和多尺度轨迹的视频质量指标（EM-VQM）。首先，密集运动轨迹被用作选择时间敏感区域的代理，在这些区域中，局部几何失真可能会显著降低感知质量。之后，通过计算1）使用弹性度量的运动轨迹变形量和2）时空结构差异，来量化时间结构不一致性和不平稳视点转换的程度。根据在两个FTV视频数据集上的综合实验结果，与最佳指标相比，所提出的指标在两个数据集上的中值皮尔逊线性相关系数值方面分别显著优于为自由视点视频设计的现有最佳指标，增益分别为12.86%和16.75%。
