Wu Gaochang, Wang Yingqian, Liu Yebin, Fang Lu, Chai Tianyou
IEEE Trans Image Process. 2021;30:8999-9013. doi: 10.1109/TIP.2021.3122089. Epub 2021 Nov 2.
Typical learning-based light field reconstruction methods demand in constructing a large receptive field by deepening their networks to capture correspondences between input views. In this paper, we propose a spatial-angular attention network to perceive non-local correspondences in the light field, and reconstruct high angular resolution light field in an end-to-end manner. Motivated by the non-local attention mechanism (Wang et al., 2018; Zhang et al., 2019), a spatial-angular attention module specifically for the high-dimensional light field data is introduced to compute the response of each query pixel from all the positions on the epipolar plane, and generate an attention map that captures correspondences along the angular dimension. Then a multi-scale reconstruction structure is proposed to efficiently implement the non-local attention in the low resolution feature space, while also preserving the high frequency components in the high-resolution feature space. Extensive experiments demonstrate the superior performance of the proposed spatial-angular attention network for reconstructing sparsely-sampled light fields with Non-Lambertian effects.
典型的基于学习的光场重建方法需要通过加深网络来构建大的感受野，以捕捉输入视图之间的对应关系。在本文中，我们提出了一种空间角注意力网络，用于感知光场中的非局部对应关系，并以端到端的方式重建高角分辨率光场。受非局部注意力机制（Wang等人，2018年；Zhang等人，2019年）的启发，引入了一个专门针对高维光场数据的空间角注意力模块，以计算来自极平面上所有位置的每个查询像素的响应，并生成一个捕获沿角维度对应关系的注意力图。然后提出了一种多尺度重建结构，以在低分辨率特征空间中有效地实现非局部注意力，同时在高分辨率特征空间中保留高频分量。大量实验证明了所提出的空间角注意力网络在重建具有非朗伯效应的稀疏采样光场方面的优越性能。
