Xu Chunyan, Wei Li, Cui Zhen, Zhang Tong, Yang Jian
IEEE Trans Image Process. 2021;30:4760-4772. doi: 10.1109/TIP.2021.3075086. Epub 2021 May 5.
The task of video object segmentation is a fundamental but challenging problem in the field of computer vision. To deal with large variations in target objects and background clutter, we propose an online adaptive video object segmentation (VOS) framework, named Meta-VOS, that learns to adapt the target-specific segmentation. Meta-VOS builds an online adaptive learning process by exploiting cumulative expertise after searching for confidence patterns across different videos/frames, and then dynamically improves the model learning from two aspects: Meta-seg learner (i.e., module updating) and Meta-seg criterion (i.e., rule of expertise). As our goal is to rapidly determine which patterns best represent the essential characteristics of specific targets in a video, Meta-seg learner is introduced to adaptively learn to update the parameters and hyperparameters of segmentation network in very few gradient descent steps. Furthermore, a Meta-seg criterion of learned expertise, which is constructed to evaluate the Meta-seg learner for the online adaptation of the segmentation network, can confidently online update positive/negative patterns under the guidance of motion cues, object appearances and learned knowledge. Comprehensive evaluations on several benchmark datasets demonstrate the superiority of our proposed Meta-VOS when compared with other state-of-the-art methods applied to the VOS problem.
视频对象分割任务是计算机视觉领域中一个基本但具有挑战性的问题。为了应对目标对象的巨大变化和背景杂波，我们提出了一种在线自适应视频对象分割（VOS）框架，名为Meta-VOS，它能够学习适应特定目标的分割。Meta-VOS通过在不同视频/帧中搜索置信模式后利用累积的专业知识构建一个在线自适应学习过程，然后从两个方面动态改进模型学习：元分割学习器（即模块更新）和元分割准则（即专业知识规则）。由于我们的目标是快速确定哪些模式最能代表视频中特定目标的基本特征，因此引入了元分割学习器，以便在极少的梯度下降步骤中自适应地学习更新分割网络的参数和超参数。此外，一个基于所学专业知识的元分割准则，用于评估元分割学习器对分割网络的在线适应性，能够在运动线索、对象外观和所学知识的指导下自信地在线更新正/负模式。在几个基准数据集上的综合评估表明，与应用于VOS问题的其他现有最先进方法相比，我们提出的Meta-VOS具有优越性。
