Yi Ran, Hu Teng, Xia Mengfei, Tang Yizhe, Liu Yong-Jin
IEEE Trans Pattern Anal Mach Intell. 2024 Dec;46(12):9975-9990. doi: 10.1109/TPAMI.2024.3432529. Epub 2024 Nov 6.
Generative Adversarial Networks have achieved significant advancements in generating and editing high-resolution images. However, most methods suffer from either requiring extensive labeled datasets or strong prior knowledge. It is also challenging for them to disentangle correlated attributes with few-shot data. In this paper, we propose FEditNet++, a GAN-based approach to explore latent semantics. It aims to enable attribute editing with limited labeled data and disentangle the correlated attributes. We propose a layer-wise feature contrastive objective, which takes into consideration content consistency and facilitates the invariance of the unrelated attributes before and after editing. Furthermore, we harness the knowledge from the pretrained discriminative model to prevent overfitting. In particular, to solve the entanglement problem between the correlated attributes from data and semantic latent correlation, we extend our model to jointly optimize multiple attributes and propose a novel decoupling loss and cross-assessment loss to disentangle them from both latent and image space. We further propose a novel-attribute disentanglement strategy to enable editing of novel attributes with unknown entanglements. Finally, we extend our model to accurately edit the fine-grained attributes. Qualitative and quantitative assessments demonstrate that our method outperforms state-of-the-art approaches across various datasets, including CelebA-HQ, RaFD, Danbooru2018 and LSUN Church.
生成对抗网络在生成和编辑高分辨率图像方面取得了重大进展。然而,大多数方法要么需要大量的标记数据集,要么需要强大的先验知识。对于它们来说,利用少样本数据解开相关属性也具有挑战性。在本文中,我们提出了FEditNet++,一种基于GAN的探索潜在语义的方法。它旨在使用有限的标记数据进行属性编辑,并解开相关属性。我们提出了一种逐层特征对比目标,该目标考虑了内容一致性,并促进了编辑前后无关属性的不变性。此外,我们利用预训练判别模型的知识来防止过拟合。特别是,为了解决数据中相关属性与语义潜在相关性之间的纠缠问题,我们扩展了模型以联合优化多个属性,并提出了一种新颖的解耦损失和交叉评估损失,以从潜在空间和图像空间中解开它们。我们进一步提出了一种新颖属性解缠策略,以实现对具有未知纠缠的新颖属性的编辑。最后,我们扩展了模型以准确编辑细粒度属性。定性和定量评估表明,我们的方法在包括CelebA-HQ、RaFD、Danbooru2018和LSUN教堂在内的各种数据集上优于现有方法。