Suppr超能文献

门控 GAN:用于多集合风格迁移的对抗门控网络。

Gated-GAN: Adversarial Gated Networks for Multi-Collection Style Transfer.

出版信息

IEEE Trans Image Process. 2019 Feb;28(2):546-560. doi: 10.1109/TIP.2018.2869695. Epub 2018 Sep 12.

Abstract

Style transfer describes the rendering of an image's semantic content as different artistic styles. Recently, generative adversarial networks (GANs) have emerged as an effective approach in style transfer by adversarially training the generator to synthesize convincing counterfeits. However, traditional GAN suffers from the mode collapse issue, resulting in unstable training and making style transfer quality difficult to guarantee. In addition, the GAN generator is only compatible with one style, so a series of GANs must be trained to provide users with choices to transfer more than one kind of style. In this paper, we focus on tackling these challenges and limitations to improve style transfer. We propose adversarial gated networks (Gated-GAN) to transfer multiple styles in a single model. The generative networks have three modules: an encoder, a gated transformer, and a decoder. Different styles can be achieved by passing input images through different branches of the gated transformer. To stabilize training, the encoder and decoder are combined as an auto-encoder to reconstruct the input images. The discriminative networks are used to distinguish whether the input image is a stylized or genuine image. An auxiliary classifier is used to recognize the style categories of transferred images, thereby helping the generative networks generate images in multiple styles. In addition, Gated-GAN makes it possible to explore a new style by investigating styles learned from artists or genres. Our extensive experiments demonstrate the stability and effectiveness of the proposed model for multi-style transfer.

摘要

风格迁移描述了将图像的语义内容渲染为不同艺术风格的过程。最近,生成对抗网络(GAN)作为一种有效的风格迁移方法出现了,通过对抗性训练生成器来合成逼真的伪造品。然而,传统的 GAN 存在模式崩溃问题,导致训练不稳定,难以保证风格迁移的质量。此外,GAN 生成器仅与一种风格兼容,因此必须训练一系列 GAN 为用户提供选择,以实现多种风格的转换。在本文中,我们专注于解决这些挑战和限制,以改进风格迁移。我们提出了对抗门控网络(Gated-GAN),以在单个模型中转换多种风格。生成网络有三个模块:编码器、门控转换器和解码器。通过将门控转换器的不同分支输入图像,可以实现不同的风格。为了稳定训练,编码器和解码器被组合为自动编码器来重建输入图像。鉴别网络用于区分输入图像是经过风格化的还是真实的图像。辅助分类器用于识别转换图像的风格类别,从而帮助生成网络生成多种风格的图像。此外,Gated-GAN 使得通过研究艺术家或流派所学习的风格来探索新的风格成为可能。我们的广泛实验证明了所提出的模型对于多风格转换的稳定性和有效性。

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验