Xu Yue, Liu Runze, Zhu Dongchen, Chen Lili, Zhang Xiaolin, Li Jiamao
Shanghai Institute of Microsystem and Information Technology, Chinese Academy of Sciences, Shanghai, China.
School of Information Science and Technology, ShanghaiTech University, Shanghai, China.
Front Neurorobot. 2024 Oct 21;18:1489021. doi: 10.3389/fnbot.2024.1489021. eCollection 2024.
Panoptic segmentation plays a crucial role in enabling robots to comprehend their surroundings, providing fine-grained scene understanding information for robots' intelligent tasks. Although existing methods have made some progress, they are prone to fail in areas with weak textures, small objects, etc. Inspired by biological vision research, we propose a cascaded contour-enhanced panoptic segmentation network called CCPSNet, attempting to enhance the discriminability of instances through structural knowledge. To acquire the scene structure, a cascade contour detection stream is designed, which extracts comprehensive scene contours using channel regulation structural perception module and coarse-to-fine cascade strategy. Furthermore, the contour-guided multi-scale feature enhancement stream is developed to boost the discrimination ability for small objects and weak textures. The stream integrates contour information and multi-scale context features through structural-aware feature modulation module and inverse aggregation technique. Experimental results show that our method improves accuracy on the Cityscapes (61.2 PQ) and COCO (43.5 PQ) datasets while also demonstrating robustness in challenging simulated real-world complex scenarios faced by robots, such as dirty cameras and rainy conditions. The proposed network promises to help the robot perceive the real scene. In future work, an unsupervised training strategy for the network could be explored to reduce the training cost.
全景分割在使机器人理解周围环境方面起着至关重要的作用，为机器人的智能任务提供细粒度的场景理解信息。尽管现有方法已经取得了一些进展，但它们在纹理较弱、物体较小等区域容易失败。受生物视觉研究的启发，我们提出了一种名为CCPSNet的级联轮廓增强全景分割网络，试图通过结构知识增强实例的可辨别性。为了获取场景结构，设计了一个级联轮廓检测流，它使用通道调节结构感知模块和由粗到细的级联策略提取综合场景轮廓。此外，还开发了轮廓引导的多尺度特征增强流，以提高对小物体和弱纹理的辨别能力。该流通过结构感知特征调制模块和逆聚合技术集成轮廓信息和多尺度上下文特征。实验结果表明，我们的方法在Cityscapes（61.2 PQ）和COCO（43.5 PQ）数据集上提高了准确率，同时在机器人面临的具有挑战性的模拟真实世界复杂场景（如脏相机和雨天条件）中也表现出了鲁棒性。所提出的网络有望帮助机器人感知真实场景。在未来的工作中，可以探索该网络的无监督训练策略以降低训练成本。
