Qi Lu, Wang Yi, Chen Yukang, Chen Ying-Cong, Zhang Xiangyu, Sun Jian, Jia Jiaya
IEEE Trans Pattern Anal Mach Intell. 2022 Oct;44(10):6377-6392. doi: 10.1109/TPAMI.2021.3085295. Epub 2022 Sep 14.
In this paper, we explore the mask representation in instance segmentation with Point-of-Interest (PoI) features. Differentiating multiple potential instances within a single PoI feature is challenging, because learning a high-dimensional mask feature for each instance using vanilla convolution demands a heavy computing burden. To address this challenge, we propose an instance-aware convolution. It decomposes this mask representation learning task into two tractable modules as instance-aware weights and instance-agnostic features. The former is to parametrize convolution for producing mask features corresponding to different instances, improving mask learning efficiency by avoiding employing several independent convolutions. Meanwhile, the latter serves as mask templates in a single point. Together, instance-aware mask features are computed by convolving the template with dynamic weights, used for the mask prediction. Along with instance-aware convolution, we propose PointINS, a simple and practical instance segmentation approach, building upon dense one-stage detectors. Through extensive experiments, we evaluated the effectiveness of our framework built upon RetinaNet and FCOS. PointINS in ResNet101 backbone achieves a 38.3 mask mean average precision (mAP) on COCO dataset, outperforming existing point-based methods by a large margin. It gives a comparable performance to the region-based Mask R-CNN K. He, G. Gkioxari, P. Dollár, and R. Girshick, "Mask R-CNN," in Proc. IEEE Int. Conf. Comput. Vis., 2017, pp. 2980-2988 with faster inference.
在本文中，我们探索了利用兴趣点（PoI）特征进行实例分割的掩码表示。在单个PoI特征内区分多个潜在实例具有挑战性，因为使用普通卷积为每个实例学习高维掩码特征需要繁重的计算负担。为应对这一挑战，我们提出了一种实例感知卷积。它将此掩码表示学习任务分解为两个易于处理的模块，即实例感知权重和实例无关特征。前者用于对卷积进行参数化，以生成对应于不同实例的掩码特征，通过避免使用多个独立卷积来提高掩码学习效率。同时，后者作为单个点处的掩码模板。通过将模板与动态权重进行卷积来计算实例感知掩码特征，用于掩码预测。连同实例感知卷积，我们提出了PointINS，这是一种基于密集单阶段检测器的简单实用的实例分割方法。通过广泛的实验，我们评估了基于RetinaNet和FCOS构建的框架的有效性。在ResNet101骨干网络中的PointINS在COCO数据集上实现了38.3的掩码平均精度均值（mAP），大幅优于现有的基于点的方法。它在推理速度更快的情况下，与基于区域的Mask R-CNN（K. He、G. Gkioxari、P. Dollár和R. Girshick，“Mask R-CNN”，发表于《IEEE国际计算机视觉会议论文集》，2017年，第2980 - 2988页）具有可比的性能。
