Li Zechao, Sun Yanpeng, Zhang Liyan, Tang Jinhui
IEEE Trans Pattern Anal Mach Intell. 2022 Dec;44(12):9904-9917. doi: 10.1109/TPAMI.2021.3132068. Epub 2022 Nov 7.
Contextual information has been shown to be powerful for semantic segmentation. This work proposes a novel Context-based Tandem Network (CTNet) by interactively exploring the spatial contextual information and the channel contextual information, which can discover the semantic context for semantic segmentation. Specifically, the Spatial Contextual Module (SCM) is leveraged to uncover the spatial contextual dependency between pixels by exploring the correlation between pixels and categories. Meanwhile, the Channel Contextual Module (CCM) is introduced to learn the semantic features including the semantic feature maps and class-specific features by modeling the long-term semantic dependence between channels. The learned semantic features are utilized as the prior knowledge to guide the learning of SCM, which can make SCM obtain more accurate long-range spatial dependency. Finally, to further improve the performance of the learned representations for semantic segmentation, the results of the two context modules are adaptively integrated to achieve better results. Extensive experiments are conducted on four widely-used datasets, i.e., PASCAL-Context, Cityscapes, ADE20K and PASCAL VOC2012. The results demonstrate the superior performance of the proposed CTNet by comparison with several state-of-the-art methods. The source code and models are available at https://github.com/syp2ysy/CTNet.
上下文信息已被证明对语义分割具有强大作用。这项工作通过交互式探索空间上下文信息和通道上下文信息，提出了一种新颖的基于上下文的串联网络（CTNet），该网络能够发现用于语义分割的语义上下文。具体而言，利用空间上下文模块（SCM）通过探索像素与类别之间的相关性来揭示像素之间的空间上下文依赖性。同时，引入通道上下文模块（CCM），通过对通道之间的长期语义依赖性进行建模来学习包括语义特征图和特定类别特征在内的语义特征。将学习到的语义特征用作先验知识来指导SCM的学习，这可以使SCM获得更准确的远距离空间依赖性。最后，为了进一步提高用于语义分割的学习表示的性能，将两个上下文模块的结果进行自适应集成以获得更好的结果。在四个广泛使用的数据集上进行了大量实验，即PASCAL-Context、Cityscapes、ADE20K和PASCAL VOC2012。结果表明，与几种最新方法相比，所提出的CTNet具有卓越的性能。源代码和模型可在https://github.com/syp2ysy/CTNet上获取。
