Wan Jia, Zhang Kaihao, Li Hongdong, Chan Antoni B
IEEE Trans Image Process. 2021;30:5085-5095. doi: 10.1109/TIP.2021.3071711. Epub 2021 May 20.
Automatic hand-drawn sketch recognition is an important task in computer vision. However, the vast majority of prior works focus on exploring the power of deep learning to achieve better accuracy on complete and clean sketch images, and thus fail to achieve satisfactory performance when applied to incomplete or destroyed sketch images. To address this problem, we first develop two datasets that contain different levels of scrawl and incomplete sketches. Then, we propose an angular-driven feedback restoration network (ADFRNet), which first detects the imperfect parts of a sketch and then refines them into high quality images, to boost the performance of sketch recognition. By introducing a novel "feedback restoration loop" to deliver information between the middle stages, the proposed model can improve the quality of generated sketch images while avoiding the extra memory cost associated with popular cascading generation schemes. In addition, we also employ a novel angular-based loss function to guide the refinement of sketch images and learn a powerful discriminator in the angular space. Extensive experiments conducted on the proposed imperfect sketch datasets demonstrate that the proposed model is able to efficiently improve the quality of sketch images and achieve superior performance over the current state-of-the-art methods.
自动手绘草图识别是计算机视觉中的一项重要任务。然而，绝大多数先前的工作都集中在探索深度学习的能力，以在完整且清晰的草图图像上实现更高的准确率，因此在应用于不完整或损坏的草图图像时未能取得令人满意的性能。为了解决这个问题，我们首先开发了两个包含不同程度潦草和不完整草图的数据集。然后，我们提出了一种角度驱动的反馈恢复网络（ADFRNet），该网络首先检测草图的不完美部分，然后将其细化为高质量图像，以提高草图识别的性能。通过引入一种新颖的“反馈恢复循环”在中间阶段传递信息，所提出的模型可以提高生成的草图图像的质量，同时避免与流行的级联生成方案相关的额外内存成本。此外，我们还采用了一种新颖的基于角度的损失函数来指导草图图像的细化，并在角度空间中学习一个强大的判别器。在所提出的不完美草图数据集上进行的大量实验表明，所提出的模型能够有效地提高草图图像的质量，并在性能上优于当前的最先进方法。
