Hung Shih-Kai, Gan John Q
School of Computer Science and Electronic Engineering, University of Essex, Colchester, United Kingdom.
PeerJ Comput Sci. 2021 Nov 17;7:e760. doi: 10.7717/peerj-cs.760. eCollection 2021.
Image data collection and labelling is costly or difficult in many real applications. Generating diverse and controllable images using conditional generative adversarial networks (GANs) for data augmentation from a small dataset is promising but challenging as deep convolutional neural networks need a large training dataset to achieve reasonable performance in general. However, unlabeled and incomplete features ( unintegral edges, simplified lines, hand-drawn sketches, discontinuous geometry shapes, etc.) can be conveniently obtained through pre-processing the training images and can be used for image data augmentation. This paper proposes a conditional GAN framework for facial image augmentation using a very small training dataset and incomplete or modified edge features as conditional input for diversity. The proposed method defines a new domain or space for refining interim images to prevent overfitting caused by using a very small training dataset and enhance the tolerance of distortions caused by incomplete edge features, which effectively improves the quality of facial image augmentation with diversity. Experimental results have shown that the proposed method can generate high-quality images of good diversity when the GANs are trained using very sparse edges and a small number of training samples. Compared to the state-of-the-art edge-to-image translation methods that directly convert sparse edges to images, when using a small training dataset, the proposed conditional GAN framework can generate facial images with desirable diversity and acceptable distortions for dataset augmentation and significantly outperform the existing methods in terms of the quality of synthesised images, evaluated by Fréchet Inception Distance (FID) and Kernel Inception Distance (KID) scores.
在许多实际应用中，图像数据的收集和标注成本高昂或困难重重。使用条件生成对抗网络（GAN）从小型数据集中生成用于数据增强的多样且可控的图像很有前景，但也颇具挑战，因为一般而言深度卷积神经网络需要大量训练数据集才能实现合理的性能。然而，通过对训练图像进行预处理，可以方便地获取未标注且不完整的特征（不完整的边缘、简化的线条、手绘草图、不连续的几何形状等），并将其用于图像数据增强。本文提出了一种条件GAN框架，用于面部图像增强，该框架使用非常小的训练数据集以及不完整或经过修改的边缘特征作为条件输入以实现多样性。所提出的方法定义了一个新的域或空间来细化中间图像，以防止因使用非常小的训练数据集而导致的过拟合，并提高对由不完整边缘特征引起的失真的容忍度，从而有效地提高了具有多样性的面部图像增强的质量。实验结果表明，当使用非常稀疏的边缘和少量训练样本训练GAN时，所提出的方法可以生成具有良好多样性的高质量图像。与直接将稀疏边缘转换为图像的现有边缘到图像翻译方法相比，当使用小型训练数据集时，所提出的条件GAN框架可以生成具有理想多样性和可接受失真的面部图像用于数据集增强，并且在合成图像质量方面显著优于现有方法，这通过弗雷歇因距离（FID）和核因距离（KID）分数进行评估。
