Yang Shuai, Wang Zhangyang, Liu Jiaying
IEEE Trans Pattern Anal Mach Intell. 2022 Jul;44(7):3807-3820. doi: 10.1109/TPAMI.2021.3055211. Epub 2022 Jun 3.
Dynamic artistic text style transfer aims to migrate the style in terms of both the appearance and motion patterns from a reference style video to the target text to create artistic text animation. Recent researches have improved the usability of transfer models by introducing texture control. However, it remains an important open challenge to investigate the control of the stylistic degree with respect to shape deformation. In this paper, we explore a new problem of dynamic artistic text style transfer with glyph stylistic degree control. The key idea is to build multi-scale glyph-style shape mappings through a novel bidirectional shape matching framework. Following this idea, we first introduce a scale-ware Shape-Matching GAN to learn such mappings to simultaneously model the style shape features at multiple scales and transfer them onto the target glyph. Furthermore, an advanced Shape-Matching GAN++ is proposed to animate a static text image based on the reference style video. Our Shape-Matching GAN++ characterizes the short-term consistency of motion patterns via shape matchings within consecutive frames, which are propagated to achieve effective long-term consistency. Experiments show that the proposed method outperforms previous state-of-the-arts both qualitatively and quantitatively, and generate high-quality and controllable artistic text.
动态艺术文本风格迁移旨在从参考风格视频中迁移外观和运动模式方面的风格到目标文本，以创建艺术文本动画。最近的研究通过引入纹理控制提高了迁移模型的可用性。然而，研究形状变形方面的风格程度控制仍然是一个重要的开放性挑战。在本文中，我们探索了具有字形风格程度控制的动态艺术文本风格迁移这一新问题。关键思想是通过一个新颖的双向形状匹配框架构建多尺度字形风格形状映射。遵循这一思想，我们首先引入一个尺度感知形状匹配生成对抗网络来学习这种映射，以同时在多个尺度上对风格形状特征进行建模并将它们转移到目标字形上。此外，还提出了一种先进的形状匹配生成对抗网络++，用于基于参考风格视频对静态文本图像进行动画处理。我们的形状匹配生成对抗网络++通过连续帧内的形状匹配来表征运动模式的短期一致性，并将其传播以实现有效的长期一致性。实验表明，所提出的方法在定性和定量方面均优于先前的最先进方法，并生成高质量且可控的艺术文本。
