Xiang Kaite, Yang Kailun, Wang Kaiwei
Opt Express. 2021 Feb 15;29(4):4802-4820. doi: 10.1364/OE.416130.
Semantic segmentation (SS) is promising for outdoor scene perception in safety-critical applications like autonomous vehicles, assisted navigation and so on. However, traditional SS is primarily based on RGB images, which limits the reliability of SS in complex outdoor scenes, where RGB images lack necessary information dimensions to fully perceive unconstrained environments. As a preliminary investigation, we examine SS in an unexpected obstacle detection scenario, which demonstrates the necessity of multimodal fusion. Thereby, in this work, we present EAFNet, an Efficient Attention-bridged Fusion Network, to exploit complementary information coming from different optical sensors. Specifically, we incorporate polarization sensing to obtain supplementary information, considering its optical characteristics for robust representation of diverse materials. By using a single-shot polarization sensor, we build the first RGB-P dataset which consists of 394 annotated pixel-aligned RGB-polarization images. A comprehensive variety of experiments shows the effectiveness of EAFNet to fuse polarization and RGB information, as well as its flexibility to be adapted to other sensor combination scenarios.
语义分割（SS）在诸如自动驾驶车辆、辅助导航等安全关键应用中的户外场景感知方面具有广阔前景。然而，传统的语义分割主要基于RGB图像，这限制了其在复杂户外场景中的可靠性，因为在这些场景中RGB图像缺乏充分感知无约束环境所需的必要信息维度。作为一项初步研究，我们在意外障碍物检测场景中研究语义分割，这证明了多模态融合的必要性。因此，在这项工作中，我们提出了EAFNet，一种高效的注意力桥接融合网络，以利用来自不同光学传感器的互补信息。具体而言，考虑到偏振传感对不同材料的鲁棒表示的光学特性，我们将其纳入以获取补充信息。通过使用单镜头偏振传感器，我们构建了第一个RGB-P数据集，该数据集由394张带注释的像素对齐RGB-偏振图像组成。各种各样的实验表明了EAFNet融合偏振和RGB信息的有效性，以及其适应其他传感器组合场景的灵活性。
