Jin Wen-Da, Xu Jun, Han Qi, Zhang Yi, Cheng Ming-Ming
IEEE Trans Image Process. 2021;30:3376-3390. doi: 10.1109/TIP.2021.3060167. Epub 2021 Mar 9.
Current RGB-D salient object detection (SOD) methods utilize the depth stream as complementary information to the RGB stream. However, the depth maps are usually of low-quality in existing RGB-D SOD datasets. Most RGB-D SOD networks trained with these datasets would produce error-prone results. In this paper, we propose a novel Complementary Depth Network (CDNet) to well exploit saliency-informative depth features for RGB-D SOD. To alleviate the influence of low-quality depth maps to RGB-D SOD, we propose to select saliency-informative depth maps as the training targets and leverage RGB features to estimate meaningful depth maps. Besides, to learn robust depth features for accurate prediction, we propose a new dynamic scheme to fuse the depth features extracted from the original and estimated depth maps with adaptive weights. What's more, we design a two-stage cross-modal feature fusion scheme to well integrate the depth features with the RGB ones, further improving the performance of our CDNet on RGB-D SOD. Experiments on seven benchmark datasets demonstrate that our CDNet outperforms state-of-the-art RGB-D SOD methods. The code is publicly available at https://github.com/blanclist/CDNet.
当前的RGB-D显著目标检测(SOD)方法将深度流用作RGB流的补充信息。然而,在现有的RGB-D SOD数据集中,深度图的质量通常较低。大多数使用这些数据集训练的RGB-D SOD网络会产生容易出错的结果。在本文中,我们提出了一种新颖的互补深度网络(CDNet),以充分利用用于RGB-D SOD的显著信息深度特征。为了减轻低质量深度图对RGB-D SOD的影响,我们建议选择显著信息深度图作为训练目标,并利用RGB特征来估计有意义的深度图。此外,为了学习用于准确预测的鲁棒深度特征,我们提出了一种新的动态方案,以自适应权重融合从原始深度图和估计深度图中提取的深度特征。更重要的是,我们设计了一种两阶段跨模态特征融合方案,以将深度特征与RGB特征很好地整合,进一步提高我们的CDNet在RGB-D SOD上的性能。在七个基准数据集上的实验表明,我们的CDNet优于现有的RGB-D SOD方法。代码可在https://github.com/blanclist/CDNet上公开获取。