IEEE Trans Image Process. 2022;31:2027-2039. doi: 10.1109/TIP.2022.3147981. Epub 2022 Feb 25.
Quality assessment of 3D-synthesized images has traditionally been based on detecting specific categories of distortions such as stretching, black-holes, blurring, etc. However, such approaches have limitations in accurately detecting distortions entirely in 3D synthesized images affecting their performance. This work proposes an algorithm to efficiently detect the distortions and subsequently evaluate the perceptual quality of 3D synthesized images. The process of generation of 3D synthesized images produces a few pixel shift between reference and 3D synthesized image, and hence they are not properly aligned with each other. To address this, we propose using morphological operation (opening) in the residual image to reduce perceptually unimportant information between the reference and the distorted 3D synthesized image. The residual image suppresses the perceptually unimportant information and highlights the geometric distortions which significantly affect the overall quality of 3D synthesized images. We utilized the information present in the residual image to quantify the perceptual quality measure and named this algorithm as Perceptually Unimportant Information Reduction (PU-IR) algorithm. At the same time, the residual image cannot capture the minor structural and geometric distortions due to the usage of erosion operation. To address this, we extract the perceptually important deep features from the pre-trained VGG-16 architectures on the Laplacian pyramid. The distortions in 3D synthesized images are present in patches, and the human visual system perceives even the small levels of these distortions. With this view, to compare these deep features between reference and distorted image, we propose using cosine similarity and named this algorithm as Deep Features extraction and comparison using Cosine Similarity (DF-CS) algorithm. The cosine similarity is based upon their similarity rather than computing the magnitude of the difference of deep features. Finally, the pooling is done to obtain the objective quality scores using simple multiplication to both PU-IR and DF-CS algorithms. Our source code is available online: https://github.com/sadbhawnathakur/3D-Image-Quality-Assessment.
三维合成图像的质量评估传统上基于检测特定类别的失真，例如拉伸、黑洞、模糊等。然而，这些方法在准确检测完全影响其性能的三维合成图像中的失真方面存在局限性。这项工作提出了一种算法，用于有效地检测失真，随后评估三维合成图像的感知质量。生成三维合成图像的过程会在参考图像和三维合成图像之间产生一些像素移位，因此它们不能很好地对齐。为了解决这个问题，我们建议在残差图像中使用形态学操作（开运算）来减少参考图像和失真的三维合成图像之间的感知不重要信息。残差图像抑制了感知不重要的信息，并突出了对三维合成图像整体质量有重大影响的几何失真。我们利用残差图像中的信息来量化感知质量度量，并将此算法命名为感知不重要信息减少（PU-IR）算法。同时，由于使用侵蚀操作，残差图像无法捕获由于使用侵蚀操作而导致的较小结构和几何失真。为了解决这个问题，我们从预先训练的 VGG-16 架构在拉普拉斯金字塔上提取感知重要的深度特征。三维合成图像中的失真以补丁的形式存在，人类视觉系统甚至可以感知到这些小级别的失真。有鉴于此，为了在参考图像和失真图像之间比较这些深度特征，我们建议使用余弦相似度，并将此算法命名为使用余弦相似度的深度特征提取和比较（DF-CS）算法。余弦相似度是基于它们的相似性，而不是计算深度特征的差异的大小。最后，使用池化对 PU-IR 和 DF-CS 算法都进行简单乘法运算，以获得客观的质量得分。我们的源代码可在网上获得：https://github.com/sadbhawnathakur/3D-Image-Quality-Assessment。
