Feng Liangjun, Zhao Chunhui
IEEE Trans Neural Netw Learn Syst. 2021 Jun;32(6):2506-2520. doi: 10.1109/TNNLS.2020.3006322. Epub 2021 Jun 2.
Zero-shot learning (ZSL) is a successful paradigm for categorizing objects from the previously unseen classes. However, it suffers from severe performance degradation in the generalized ZSL (GZSL) setting, i.e., to recognize the test images that are from both seen and unseen classes. In this article, we present a simple but effective mechanism for GZSL and more open scenarios based on a transfer-increment strategy. On the one hand, a dual-knowledge-source-based generative model is constructed to tackle the missing data problem. Specifically, the local relational knowledge extracted from the label-embedding space and the global relational knowledge, which is the estimated data center in the feature-embedding space, are concurrently considered to synthesize the virtual exemplars. On the other hand, we further explore the training issue for the generative models under the GZSL setting. Two incremental training modes are designed to learn directly the unseen classes from the synthesized exemplars instead of the training classifiers with the seen and synthesized unseen exemplars together. It not only presents an effective unseen class learning but also requires less computing and storage resources in practical application. Comprehensive experiments are conducted based on five benchmark data sets. In comparison with the state-of-the-art methods, both the generating and training processes are considered for virtual exemplars by the proposed transfer-increment strategy, which results in a significant improvement in the conventional and GZSL tasks.
零样本学习(ZSL)是一种用于对来自以前未见过的类别的对象进行分类的成功范式。然而,在广义零样本学习(GZSL)设置中,即在识别来自已见和未见类别的测试图像时,它会遭受严重的性能下降。在本文中,我们基于转移增量策略提出了一种简单而有效的机制,用于GZSL和更开放的场景。一方面,构建了一个基于双知识源的生成模型来解决缺失数据问题。具体来说,从标签嵌入空间中提取的局部关系知识和在特征嵌入空间中估计的数据中心这一全局关系知识被同时考虑,以合成虚拟样本。另一方面,我们进一步探讨了GZSL设置下生成模型的训练问题。设计了两种增量训练模式,直接从合成样本中学习未见类别,而不是将已见和合成的未见样本一起用于训练分类器。这不仅提出了一种有效的未见类别学习方法,而且在实际应用中需要更少的计算和存储资源。基于五个基准数据集进行了全面的实验。与现有方法相比,所提出的转移增量策略在虚拟样本的生成和训练过程中都进行了考虑,这在传统任务和GZSL任务中都带来了显著的改进。