Zhou Wenhui, Lin Lili, Hong Yongjie, Li Qiujian, Shen Xingfa, Kuruoglu Ercan Engin
IEEE Trans Neural Netw Learn Syst. 2024 Nov;35(11):15660-15674. doi: 10.1109/TNNLS.2023.3289056. Epub 2024 Oct 29.
Although learning-based light field disparity estimation has achieved great progress in the most recent years, the performance of unsupervised light field learning is still hindered by occlusions and noises. By analyzing the overall strategy underlying the unsupervised methodology and the light field geometry implied in epipolar plane images (EPIs), we look beyond the photometric consistency assumption, and design an occlusion-aware unsupervised framework to deal with the situations of photometric consistency conflict. Specifically, we present a geometry-based light field occlusion modeling, which predicts a group of visibility masks and occlusion maps, respectively, by forward warping and backward EPI-line tracing. In order to learn better the noise- and occlusion-invariant representations of the light field, we propose two occlusion-aware unsupervised losses: occlusion-aware SSIM and statistics-based EPI loss. Experiment results demonstrate that our method can improve the estimation accuracy of light field depth over the occluded and noisy regions, and preserve the occlusion boundaries better.
尽管基于学习的光场视差估计在最近几年取得了很大进展，但无监督光场学习的性能仍然受到遮挡和噪声的阻碍。通过分析无监督方法背后的整体策略以及极平面图像（EPI）中隐含的光场几何结构，我们超越了光度一致性假设，设计了一个遮挡感知的无监督框架来处理光度一致性冲突的情况。具体来说，我们提出了一种基于几何的光场遮挡建模方法，通过前向扭曲和反向EPI线追踪分别预测一组可见性掩码和遮挡图。为了更好地学习光场的噪声和遮挡不变表示，我们提出了两种遮挡感知的无监督损失：遮挡感知结构相似性（SSIM）和基于统计的EPI损失。实验结果表明，我们的方法可以提高光场深度在遮挡和噪声区域的估计精度，并更好地保留遮挡边界。
