College of Information Science and Technology, Donghua University, Shanghai 201620, China.
Engineering Research Center of Digitized Textile & Apparel Technology, Ministry of Education, Donghua University, Shanghai 201620, China.
Sensors (Basel). 2021 Sep 22;21(19):6336. doi: 10.3390/s21196336.
Image inpainting aims to fill in corrupted regions with visually realistic and semantically plausible contents. In this paper, we propose a progressive image inpainting method, which is based on a forked-then-fused decoder network. A unit called PC-RN, which is the combination of partial convolution and region normalization, serves as the basic component to construct inpainting network. The PC-RN unit can extract useful features from the valid surroundings and can suppress incompleteness-caused interference at the same time. The forked-then-fused decoder network consists of a local reception branch, a long-range attention branch, and a squeeze-and-excitation-based fusing module. Two multi-scale contextual attention modules are deployed into the long-range attention branch for adaptively borrowing features from distant spatial positions. Progressive inpainting strategy allows the attention modules to use the previously filled region to reduce the risk of allocating wrong attention. We conduct extensive experiments on three benchmark databases: Places2, Paris StreetView, and CelebA. Qualitative and quantitative results show that the proposed inpainting model is superior to state-of-the-art works. Moreover, we perform ablation studies to reveal the functionality of each module for the image inpainting task.
图像修复旨在用视觉逼真和语义合理的内容填充损坏的区域。在本文中，我们提出了一种基于分叉-融合解码器网络的渐进式图像修复方法。一个名为 PC-RN 的单元，它是部分卷积和区域归一化的组合，作为构建图像修复网络的基本组件。PC-RN 单元可以从有效环境中提取有用的特征，同时抑制由不完整性引起的干扰。分叉-融合解码器网络由局部接收分支、远程注意分支和基于挤压和激励的融合模块组成。两个多尺度上下文注意模块被部署到远程注意分支中，以自适应地从远程空间位置借用特征。渐进式图像修复策略允许注意模块使用之前填充的区域来降低分配错误注意的风险。我们在三个基准数据库：Places2、Paris StreetView 和 CelebA 上进行了广泛的实验。定性和定量结果表明，所提出的图像修复模型优于最先进的作品。此外，我们进行了消融研究，以揭示每个模块对图像修复任务的功能。
