Roziere Baptiste, Riviere Morgane, Teytaud Olivier, Rapin Jeremy, LeCun Yann, Couprie Camille
IEEE Trans Image Process. 2021;30:4036-4045. doi: 10.1109/TIP.2021.3065845. Epub 2021 Apr 7.
The task of image generation started receiving some attention from artists and designers, providing inspiration for new creations. However, exploiting the results of deep generative models such as Generative Adversarial Networks can be long and tedious given the lack of existing tools. In this work, we propose a simple strategy to inspire creators with new generations learned from a dataset of their choice, while providing some control over the output. We design a simple optimization method to find the optimal latent parameters corresponding to the closest generation to any input inspirational image. Specifically, we allow the generation given an inspirational image of the user's choosing by performing several optimization steps to recover optimal parameters from the model's latent space. We tested several exploration methods from classical gradient descents to gradient-free optimizers. Many gradient-free optimizers just need comparisons (better/worse than another image), so they can even be used without numerical criterion nor inspirational image, only with human preferences. Thus, by iterating on one's preferences we can make robust facial composite or fashion generation algorithms. Our results on four datasets of faces, fashion images, and textures show that satisfactory images are effectively retrieved in most cases.
图像生成任务开始受到艺术家和设计师的一些关注,为新创作提供了灵感。然而,鉴于缺乏现有工具,利用生成对抗网络等深度生成模型的结果可能既漫长又乏味。在这项工作中,我们提出了一种简单的策略,用从创作者选择的数据集中学习到的新生成结果来启发创作者,同时对输出进行一定程度的控制。我们设计了一种简单的优化方法,以找到与任何输入的灵感图像最接近的生成对应的最优潜在参数。具体来说,我们通过执行几个优化步骤从模型的潜在空间中恢复最优参数,从而根据用户选择的灵感图像进行生成。我们测试了从经典梯度下降到无梯度优化器的几种探索方法。许多无梯度优化器只需要比较(比另一幅图像更好/更差),所以它们甚至可以在没有数值标准和灵感图像的情况下使用,仅根据人类偏好即可。因此,通过迭代个人偏好,我们可以制作出强大的面部合成或时尚生成算法。我们在人脸、时尚图像和纹理的四个数据集上的结果表明,在大多数情况下都能有效地检索到令人满意的图像。