Shanghai Film Academy, Shanghai University, Shanghai, PR China.
PLoS One. 2021 Nov 30;16(11):e0259953. doi: 10.1371/journal.pone.0259953. eCollection 2021.
Aiming at these problems of image colorization algorithms based on deep learning, such as color bleeding and insufficient color, this paper converts the study of image colorization to the optimization of image semantic segmentation, and proposes a fully automatic image colorization model based on semantic segmentation technology. Firstly, we use the encoder as the local feature extraction network and use VGG-16 as the global feature extraction network. These two parts do not interfere with each other, but they share the low-level feature. Then, the first fusion module is constructed to merge local features and global features, and the fusion results are input into semantic segmentation network and color prediction network respectively. Finally, the color prediction network obtains the semantic segmentation information of the image through the second fusion module, and predicts the chrominance of the image based on it. Through several sets of experiments, it is proved that the performance of our model becomes stronger and stronger under the nourishment of the data. Even in some complex scenes, our model can predict reasonable colors and color correctly, and the output effect is very real and natural.
针对基于深度学习的图像着色算法存在的颜色渗色和颜色不足等问题，本文将图像着色的研究转向图像语义分割的优化，提出了一种基于语义分割技术的全自动图像着色模型。首先，我们使用编码器作为局部特征提取网络，并使用 VGG-16 作为全局特征提取网络。这两部分互不干扰，但共享底层特征。然后，构建第一个融合模块来融合局部特征和全局特征，并将融合结果分别输入到语义分割网络和颜色预测网络中。最后，颜色预测网络通过第二个融合模块获取图像的语义分割信息，并基于该信息预测图像的色度。通过几组实验证明，我们的模型在数据的滋养下性能越来越强。即使在一些复杂的场景中，我们的模型也可以预测出合理的颜色和正确的颜色，输出效果非常真实自然。
