Zheng Haitian, Chen Lele, Xu Chenliang, Luo Jiebo
IEEE Trans Image Process. 2021;30:1898-1909. doi: 10.1109/TIP.2020.3031108. Epub 2021 Jan 20.
Pose guided synthesis aims to generate a new image in an arbitrary target pose while preserving the appearance details from the source image. Existing approaches rely on either hard-coded spatial transformations or 3D body modeling. They often overlook complex non-rigid pose deformation or unmatched occluded regions, thus fail to effectively preserve appearance information. In this article, we propose a pose flow learning scheme that learns to transfer the appearance details from the source image without resorting to annotated correspondences. Based on such learned pose flow, we proposed GarmentNet and SynthesisNet, both of which use multi-scale feature-domain alignment for coarse-to-fine synthesis. Experiments on the DeepFashion, MVC dataset and additional real-world datasets demonstrate that our approach compares favorably with the state-of-the-art methods and generalizes to unseen poses and clothing styles.
姿态引导合成旨在生成一个处于任意目标姿态的新图像，同时保留源图像的外观细节。现有方法要么依赖硬编码的空间变换，要么依赖3D人体建模。它们常常忽略复杂的非刚性姿态变形或不匹配的遮挡区域，因此无法有效地保留外观信息。在本文中，我们提出了一种姿态流学习方案，该方案无需借助注释对应关系就能学习从源图像中传递外观细节。基于这种学习到的姿态流，我们提出了服装网络（GarmentNet）和合成网络（SynthesisNet），它们都使用多尺度特征域对齐进行从粗到精的合成。在深度时尚（DeepFashion）、MVC数据集以及其他真实世界数据集上的实验表明，我们的方法与现有最先进方法相比具有优势，并且能够推广到未见过的姿态和服装风格。
