Wang Xuehao, Li Shuai, Chen Chenglizhao, Fang Yuming, Hao Aimin, Qin Hong
IEEE Trans Image Process. 2021;30:458-471. doi: 10.1109/TIP.2020.3037470. Epub 2020 Nov 23.
Existing RGB-D salient object detection methods treat depth information as an independent component to complement RGB and widely follow the bistream parallel network architecture. To selectively fuse the CNN features extracted from both RGB and depth as a final result, the state-of-the-art (SOTA) bistream networks usually consist of two independent subbranches: one subbranch is used for RGB saliency, and the other aims for depth saliency. However, depth saliency is persistently inferior to the RGB saliency because the RGB component is intrinsically more informative than the depth component. The bistream architecture easily biases its subsequent fusion procedure to the RGB subbranch, leading to a performance bottleneck. In this paper, we propose a novel data-level recombination strategy to fuse RGB with D (depth) before deep feature extraction, where we cyclically convert the original 4-dimensional RGB-D into DGB, RDB and RGD. Then, a newly lightweight designed triple-stream network is applied over these novel formulated data to achieve an optimal channel-wise complementary fusion status between the RGB and D, achieving a new SOTA performance.
现有的RGB-D显著目标检测方法将深度信息视为补充RGB的独立组件，并广泛采用双流并行网络架构。为了将从RGB和深度中提取的CNN特征选择性地融合为最终结果，当前最先进的（SOTA）双流网络通常由两个独立的子分支组成：一个子分支用于RGB显著性，另一个用于深度显著性。然而，深度显著性一直不如RGB显著性，因为RGB组件本质上比深度组件更具信息性。双流架构很容易使其后继融合过程偏向RGB子分支，从而导致性能瓶颈。在本文中，我们提出了一种新颖的数据级重组策略，在深度特征提取之前将RGB与D（深度）融合，我们将原始的4维RGB-D循环转换为DGB、RDB和RGD。然后，在这些新构建的数据上应用新设计的轻量级三流网络，以实现RGB和D之间的最佳通道级互补融合状态，从而实现新的SOTA性能。
