Department of Electronics and Informatics (ETRO), Vrije Universiteit Brussel (VUB), Pleinlaan 2, 1050 Brussels, Belgium.
Sensors (Basel). 2021 May 7;21(9):3246. doi: 10.3390/s21093246.
In this paper, we propose a novel filtering method based on deep attention networks for the quality enhancement of light field (LF) images captured by plenoptic cameras and compressed using the High Efficiency Video Coding (HEVC) standard. The proposed architecture was built using efficient complex processing blocks and novel attention-based residual blocks. The network takes advantage of the macro-pixel (MP) structure, specific to LF images, and processes each reconstructed MP in the luminance (Y) channel. The input patch is represented as a tensor that collects, from an MP neighbourhood, four Epipolar Plane Images (EPIs) at four different angles. The experimental results on a common LF image database showed high improvements over HEVC in terms of the structural similarity index (SSIM), with an average Y-Bjøntegaard Delta (BD)-rate savings of 36.57%, and an average Y-BD-PSNR improvement of 2.301 dB. Increased performance was achieved when the HEVC built-in filtering methods were skipped. The visual results illustrate that the enhanced image contains sharper edges and more texture details. The ablation study provides two robust solutions to reduce the inference time by 44.6% and the network complexity by 74.7%. The results demonstrate the potential of attention networks for the quality enhancement of LF images encoded by HEVC.
在本文中，我们提出了一种基于深度注意网络的新颖滤波方法，用于增强由光场（LF）相机捕获并使用高效视频编码（HEVC）标准压缩的 LF 图像的质量。所提出的架构使用高效的复数处理块和新颖的基于注意力的残差块构建。该网络利用 LF 图像特有的宏像素（MP）结构，对亮度（Y）通道中的每个重建 MP 进行处理。输入补丁表示为一个张量，从 MP 邻域中收集四个不同角度的四个立体图像（EPI）。在常见的 LF 图像数据库上的实验结果表明，与 HEVC 相比，在结构相似性指数（SSIM）方面有很大的提高，平均 Y-Bjøntegaard Delta（BD）-率节省 36.57%，平均 Y-BD-PSNR 提高 2.301dB。当跳过 HEVC 内置滤波方法时，性能得到提高。视觉结果表明，增强后的图像具有更清晰的边缘和更多的纹理细节。消融研究提供了两种稳健的解决方案，可将推断时间减少 44.6%，将网络复杂度降低 74.7%。结果表明，注意网络在增强 HEVC 编码的 LF 图像质量方面具有潜力。
