IEEE Trans Pattern Anal Mach Intell. 2023 Jun;45(6):7639-7653. doi: 10.1109/TPAMI.2022.3223784. Epub 2023 May 5.
The task of Few-shot learning (FSL) aims to transfer the knowledge learned from base categories with sufficient labelled data to novel categories with scarce known information. It is currently an important research question and has great practical values in the real-world applications. Despite extensive previous efforts are made on few-shot learning tasks, we emphasize that most existing methods did not take into account the distributional shift caused by sample selection bias in the FSL scenario. Such a selection bias can induce spurious correlation between the semantic causal features, that are causally and semantically related to the class label, and the other non-causal features. Critically, the former ones should be invariant across changes in distributions, highly related to the classes of interest, and thus well generalizable to novel classes, while the latter ones are not stable to changes in the distribution. To resolve this problem, we propose a novel data augmentation strategy dubbed as PatchMix that can break this spurious dependency by replacing the patch-level information and supervision of the query images with random gallery images from different classes from the query ones. We theoretically show that such an augmentation mechanism, different from existing ones, is able to identify the causal features. To further make these features to be discriminative enough for classification, we propose Correlation-guided Reconstruction (CGR) and Hardness-Aware module for instance discrimination and easier discrimination between similar classes. Moreover, such a framework can be adapted to the unsupervised FSL scenario. The utility of our method is demonstrated on the state-of-the-art results consistently achieved on several benchmarks including miniImageNet, tieredImageNet, CIFAR-FS, CUB, Cars, Places and Plantae, in all settings of single-domain, cross-domain and unsupervised FSL. By studying the intra-variance property of learned features and visualizing the learned features, we further quantitatively and qualitatively show that such a promising result is due to the effectiveness in learning causal features.
少样本学习（FSL）的任务旨在将从具有足够标记数据的基础类别中学到的知识转移到具有稀缺已知信息的新类别中。这是当前一个重要的研究问题，在实际应用中具有很大的实用价值。尽管在少样本学习任务上已经做了大量的前期工作，但我们强调，大多数现有的方法并没有考虑到在 FSL 场景中样本选择偏差引起的分布转移。这种选择偏差会在语义因果特征和其他非因果特征之间产生虚假相关性，而这些特征与类标签在因果和语义上都有关系。至关重要的是，前者在分布变化中应该是不变的，与感兴趣的类别高度相关，因此可以很好地推广到新类别，而后者在分布变化中是不稳定的。为了解决这个问题，我们提出了一种新的数据增强策略，称为 PatchMix，可以通过用来自不同类别的查询图像的随机图库图像替换查询图像的补丁级信息和监督来打破这种虚假依赖关系。我们从理论上证明，这种与现有方法不同的增强机制能够识别因果特征。为了使这些特征在分类时更具区分性，我们提出了 Correlation-guided Reconstruction (CGR)和 Hardness-Aware 模块，用于实例判别和相似类之间的更容易判别。此外，这种框架可以适应无监督的 FSL 场景。我们的方法在几个基准上的最先进的结果中得到了一致的验证，包括 miniImageNet、tieredImageNet、CIFAR-FS、CUB、Cars、Places 和 Plantae，涵盖了单域、跨域和无监督 FSL 的所有设置。通过研究学习特征的内方差特性和可视化学习特征，我们进一步从定量和定性两个方面表明，这种有希望的结果是由于学习因果特征的有效性。
