Wang Juan, Yuan Chunfeng, Li Bing, Deng Ying, Hu Weiming, Maybank Stephen
IEEE Trans Pattern Anal Mach Intell. 2023 Oct;45(10):12377-12393. doi: 10.1109/TPAMI.2023.3284431. Epub 2023 Sep 5.
Blind image inpainting involves two critical aspects, i.e., "where to inpaint" and "how to inpaint". Knowing "where to inpaint" can eliminate the interference arising from corrupted pixel values; a good "how to inpaint" strategy yields high-quality inpainted results robust to various corruptions. In existing methods, these two aspects usually lack explicit and separate consideration. This paper fully explores these two aspects and proposes a self-prior guided inpainting network (SIN). The self-priors are obtained by detecting semantic-discontinuous regions and by predicting global semantic structures of the input image. On the one hand, the self-priors are incorporated into the SIN, which enables the SIN to perceive valid context information from uncorrupted regions and to synthesize semantic-aware textures for corrupted regions. On the other hand, the self-priors are reformulated to provide a pixel-wise adversarial feedback and a high-level semantic structure feedback, which can promote the semantic continuity of inpainted images. Experimental results demonstrate that our method achieves state-of-the-art performance in metric scores and in visual quality. It has an advantage over many existing methods that assume "where to inpaint" is known in advance. Extensive experiments on a series of related image restoration tasks validate the effectiveness of our method in obtaining high-quality inpainting.
盲图像修复涉及两个关键方面，即“在哪里修复”和“如何修复”。知道“在哪里修复”可以消除由损坏的像素值引起的干扰；一个好的“如何修复”策略会产生对各种损坏具有鲁棒性的高质量修复结果。在现有方法中，这两个方面通常缺乏明确且分开的考虑。本文充分探索了这两个方面，并提出了一种自先验引导的修复网络（SIN）。自先验是通过检测语义不连续区域和预测输入图像的全局语义结构来获得的。一方面，自先验被纳入SIN，这使得SIN能够从未损坏区域感知有效的上下文信息，并为损坏区域合成语义感知纹理。另一方面，自先验被重新制定以提供逐像素的对抗反馈和高级语义结构反馈，这可以促进修复图像的语义连续性。实验结果表明，我们的方法在度量分数和视觉质量方面都达到了当前的最优性能。与许多预先假设知道“在哪里修复”的现有方法相比，它具有优势。在一系列相关图像恢复任务上的大量实验验证了我们的方法在获得高质量修复方面的有效性。
