IEEE Trans Pattern Anal Mach Intell. 2022 Dec;44(12):10261-10269. doi: 10.1109/TPAMI.2021.3134684. Epub 2022 Nov 7.
The high computational cost of neural networks has prevented recent successes in RGB-D salient object detection (SOD) from benefiting real-world applications. Hence, this article introduces a novel network, MobileSal, which focuses on efficient RGB-D SOD using mobile networks for deep feature extraction. However, mobile networks are less powerful in feature representation than cumbersome networks. To this end, we observe that the depth information of color images can strengthen the feature representation related to SOD if leveraged properly. Therefore, we propose an implicit depth restoration (IDR) technique to strengthen the mobile networks' feature representation capability for RGB-D SOD. IDR is only adopted in the training phase and is omitted during testing, so it is computationally free. Besides, we propose compact pyramid refinement (CPR) for efficient multi-level feature aggregation to derive salient objects with clear boundaries. With IDR and CPR incorporated, MobileSal performs favorably against state-of-the-art methods on six challenging RGB-D SOD datasets with much faster speed (450fps for the input size of 320×320) and fewer parameters (6.5M). The code is released at https://mmcheng.net/mobilesal.
神经网络的计算成本很高，这使得最近在 RGB-D 显著目标检测 (SOD) 方面的成功无法应用于实际应用。因此，本文引入了一个新的网络，MobileSal，它专注于使用移动网络进行高效的 RGB-D SOD 深度特征提取。然而，移动网络在特征表示方面的能力不如繁琐的网络强大。为此，我们观察到，如果正确利用彩色图像的深度信息，可以增强与 SOD 相关的特征表示。因此，我们提出了一种隐式深度恢复 (IDR) 技术，以增强移动网络进行 RGB-D SOD 的特征表示能力。IDR 仅在训练阶段采用，在测试阶段省略，因此计算上是免费的。此外，我们提出了紧凑金字塔细化 (CPR) 技术，用于高效的多层次特征聚合，以获得具有清晰边界的显著目标。通过集成 IDR 和 CPR，MobileSal 在六个具有挑战性的 RGB-D SOD 数据集上的表现优于最先进的方法，速度更快（输入大小为 320×320 时为 450fps），参数更少（6.5M）。代码发布在 https://mmcheng.net/mobilesal。
