Zhu Conghui, Jiang Yi, Yuan Yan, Su Lijuan, Yin Xiaorui, Kong Deqian
Opt Express. 2024 Feb 12;32(4):5362-5379. doi: 10.1364/OE.506541.
The light field (LF) imaging systems face a trade-off between the spatial and angular resolution in a limited sensor resolution. Various networks have been proposed to enhance the spatial resolution of the sub-aperture image (SAI). However, the spatial shift-variant characteristics of the LF are not considered, and few efforts have been made to recover a full-resolution (FR) image. In this paper, we propose an FR image restoration method by embedding LF degradation kernels into the network. An explicit convolution model based on the scalar diffraction theory is first derived to calculate the system response and imaging matrix. Based on the analysis of LF image formation, we establish the mapping from an FR image to the SAI through the SAI kernel, which is a spatial shift-variant degradation (SSVD) kernel. Then, the SSVD kernels are embedded into the proposed network as prior knowledge. An SSVD convolution layer is specially designed to handle the view-wise degradation feature and speed up the training process. A refinement block is designed to preserve the entire image details. Moreover, our network is evaluated on extensive simulated and real-world LF images to demonstrate its superior performance compared with other methods. Experiments on a multi-focus scene further prove that our network is suitable for any in-focus or defocused conditions.
在有限的传感器分辨率下，光场（LF）成像系统面临着空间分辨率和角分辨率之间的权衡。已经提出了各种网络来提高子孔径图像（SAI）的空间分辨率。然而，尚未考虑LF的空间移位变化特性，并且在恢复全分辨率（FR）图像方面所做的努力很少。在本文中，我们提出了一种通过将LF退化核嵌入网络来恢复FR图像的方法。首先基于标量衍射理论推导了一个显式卷积模型，以计算系统响应和成像矩阵。基于对LF图像形成的分析，我们通过SAI核建立了从FR图像到SAI的映射，SAI核是一种空间移位变化退化（SSVD）核。然后，将SSVD核作为先验知识嵌入到所提出的网络中。专门设计了一个SSVD卷积层来处理逐视图退化特征并加快训练过程。设计了一个细化块来保留整个图像细节。此外，我们的网络在大量模拟和真实世界的LF图像上进行了评估，以证明其与其他方法相比的优越性能。在多焦点场景上的实验进一步证明了我们的网络适用于任何对焦或失焦条件。
