State Key Laboratory of Fluid Power & Mechatronic Systems, Zhejiang University, Hangzhou, 310027, China.
State Key Laboratory of Fluid Power & Mechatronic Systems, Zhejiang University, Hangzhou, 310027, China.
Neural Netw. 2024 Aug;176:106353. doi: 10.1016/j.neunet.2024.106353. Epub 2024 May 1.
Garment transfer can wear the garment of the model image onto the personal image. As garment transfer leverages wild and cheap garment input, it has attracted tremendous attention in the community and has a huge commercial potential. Since the ground truth of garment transfer is almost unavailable in reality, previous studies have treated garment transfer as either pose transfer or garment-pose disentanglement, and trained garment transfer in self-supervised learning, However, these implementation methods do not cover garment transfer intentions completely and face the robustness issue in the testing phase. Notably, virtual try-on technology has exhibited superior performance using self-supervised learning, we propose to supervise the garment transfer training via knowledge distillation from virtual try-on. Specifically, the overall pipeline is first to infer a garment transfer parsing, and to use it to guide downstream warping and inpainting tasks. The transfer parsing reasoning model learns the response and feature knowledge from the try-on parsing reasoning model and absorbs the hard knowledge from the ground truth. The progressive flow warping model learns the content knowledge from virtual try-on for a reasonable and precise garment warping. To enhance transfer realism, we propose an arm regrowth task to infer exposed skin. Experiments demonstrate that our method has state-of-the-art performance in transferring garments between persons compared with other virtual try-on and garment transfer methods.
服装迁移可以将模特形象的服装穿到个人形象上。由于服装迁移利用了野生和廉价的服装输入，因此在社区中引起了极大的关注，具有巨大的商业潜力。由于服装迁移的真实情况几乎无法在现实中获得，因此以前的研究将服装迁移视为姿势迁移或服装姿势解耦，并在自监督学习中进行服装迁移训练，但是这些实施方法并没有完全涵盖服装迁移意图，并且在测试阶段面临稳健性问题。值得注意的是，虚拟试穿技术使用自监督学习表现出了优异的性能，我们提出通过从虚拟试穿中进行知识蒸馏来监督服装迁移训练。具体来说，总体流程首先推断服装转移解析，然后使用它来指导下游的变形和填充任务。转移解析推理模型从试穿解析推理模型中学习响应和特征知识，并从真实数据中吸收硬知识。渐进式流变形模型从虚拟试穿中学习合理而精确的服装变形的内容知识。为了增强转移的真实性，我们提出了手臂再生任务来推断暴露的皮肤。实验表明，与其他虚拟试穿和服装迁移方法相比，我们的方法在人与人之间的服装迁移方面具有最先进的性能。
