Liu Binghao, Jiao Jianbin, Ye Qixiang
IEEE Trans Image Process. 2021;30:3142-3153. doi: 10.1109/TIP.2021.3058512. Epub 2021 Feb 25.
Few-shot semantic segmentation remains an open problem because limited support (training) images are insufficient to represent the diverse semantics within target categories. Conventional methods typically model a target category solely using information from the support image(s), resulting in incomplete semantic activation. In this paper, we propose a novel few-shot segmentation approach, termed harmonic feature activation (HFA), with the aim to implement dense support-to-query semantic transform by incorporating the features of both query and support images. HFA is formulated as a bilinear model, which takes charge of the pixel-wise dense correlation (bilinear feature activation) between query and support images in a systematic way. HFA incorporates a low-rank decomposition procedure, which speeds up bilinear feature activation with negligible performance cost. In addition, a semantic diffusion procedure is fused with HFA, which further improves the global harmony and local consistency of the feature activation. Extensive experiments on commonly used datasets (PASCAL VOC and MS COCO) show that HFA improves the state-of-the-arts with significant margins. Code is available at https://github.com/Bibikiller/HFA.
少样本语义分割仍然是一个开放问题，因为有限的支持（训练）图像不足以表示目标类别中的多样语义。传统方法通常仅使用来自支持图像的信息对目标类别进行建模，导致语义激活不完整。在本文中，我们提出了一种新颖的少样本分割方法，称为谐波特征激活（HFA），旨在通过结合查询图像和支持图像的特征来实现密集的支持到查询语义转换。HFA被公式化为一个双线性模型，它以系统的方式负责查询图像和支持图像之间的逐像素密集相关性（双线性特征激活）。HFA包含一个低秩分解过程，该过程以可忽略不计的性能成本加速双线性特征激活。此外，一个语义扩散过程与HFA融合，进一步提高了特征激活的全局协调性和局部一致性。在常用数据集（PASCAL VOC和MS COCO）上的大量实验表明，HFA显著提高了当前的最优水平。代码可在https://github.com/Bibikiller/HFA获取。
