Wang Jinghua, Cheng Ming-Ming, Jiang Jianmin
IEEE Trans Image Process. 2021;30:5505-5517. doi: 10.1109/TIP.2021.3084354. Epub 2021 Jun 11.
In learning-based image processing a model that is learned in one domain often performs poorly in another since the image samples originate from different sources and thus have different distributions. Domain adaptation techniques alleviate the problem of domain shift by learning transferable knowledge from the source domain to the target domain. Zero-shot domain adaptation (ZSDA) refers to a category of challenging tasks in which no target-domain sample for the task of interest is accessible for training. To address this challenge, we propose a simple but effective method that is based on the strategy of domain shift preservation across tasks. First, we learn the shift between the source domain and the target domain from an irrelevant task for which sufficient data samples from both domains are available. Then, we transfer the domain shift to the task of interest under the hypothesis that different tasks may share the domain shift for a specified pair of domains. Via this strategy, we can learn a model for the unseen target domain of the task of interest. Our method uses two coupled generative adversarial networks (CoGANs) to capture the joint distribution of data samples in dual-domains and another generative adversarial network (GAN) to explicitly model the domain shift. The experimental results on image classification and semantic segmentation demonstrate the satisfactory performance of our method in transferring various kinds of domain shifts across tasks.
在基于学习的图像处理中,在一个领域中学习的模型在另一个领域中通常表现不佳,因为图像样本来自不同的来源,因此具有不同的分布。域适应技术通过从源域到目标域学习可转移的知识来缓解域转移问题。零样本域适应(ZSDA)指的是一类具有挑战性的任务,在这类任务中,没有可用于训练的感兴趣任务的目标域样本。为了应对这一挑战,我们提出了一种简单但有效的方法,该方法基于跨任务保留域转移的策略。首先,我们从一个不相关的任务中学习源域和目标域之间的转移,对于该任务,两个域都有足够的数据样本。然后,在不同任务可能为指定的一对域共享域转移的假设下,我们将域转移应用到感兴趣的任务中。通过这种策略,我们可以为感兴趣任务的未见目标域学习一个模型。我们的方法使用两个耦合生成对抗网络(CoGAN)来捕获双域中数据样本的联合分布,并使用另一个生成对抗网络(GAN)来明确地对域转移进行建模。图像分类和语义分割的实验结果证明了我们的方法在跨任务转移各种域转移方面的令人满意的性能。