Yin Zi, Yiu Valentin, Hu Xiaolin, Tang Liang
School of Technology, Beijing Forestry University, Beijing, 100083 China.
Department of Computer Science and Technology,State Key Laboratory of Intelligent Technology and Systems, Institute for Artificial Intelligence,Beijing National Research Center for Information Science and Technology,THBI, Tsinghua University, Beijing, 100084 China.
Cogn Neurodyn. 2021 Feb;15(1):169-179. doi: 10.1007/s11571-020-09615-4. Epub 2020 Jul 13.
Face parsing is an important computer vision task that requires accurate pixel segmentation of facial parts (such as eyes, nose, mouth, etc.), providing a basis for further face analysis, modification, and other applications. Interlinked Convolutional Neural Networks (iCNN) was proved to be an effective two-stage model for face parsing. However, the original iCNN was trained separately in two stages, limiting its performance. To solve this problem, we introduce a simple, end-to-end face parsing framework: STN-aided iCNN(STN-iCNN), which extends the iCNN by adding a Spatial Transformer Network (STN) between the two isolated stages. The STN-iCNN uses the STN to provide a trainable connection to the original two-stage iCNN pipeline, making end-to-end joint training possible. Moreover, as a by-product, STN also provides more precise cropped parts than the original cropper. Due to these two advantages, our approach significantly improves the accuracy of the original model. Our model achieved competitive performance on the Helen Dataset, the standard face parsing dataset. It also achieved superior performance on CelebAMask-HQ dataset, proving its good generalization. Our code has been released at https://github.com/aod321/STN-iCNN.
面部解析是一项重要的计算机视觉任务，它需要对面部各个部分（如眼睛、鼻子、嘴巴等）进行精确的像素分割，为进一步的面部分析、修改及其他应用提供基础。互联卷积神经网络（iCNN）被证明是一种用于面部解析的有效的两阶段模型。然而，原始的iCNN是分两个阶段单独训练的，这限制了其性能。为了解决这个问题，我们引入了一个简单的端到端面部解析框架：基于空间变换器网络辅助的iCNN（STN-iCNN），它通过在两个独立阶段之间添加一个空间变换器网络（STN）来扩展iCNN。STN-iCNN使用STN为原始的两阶段iCNN管道提供一个可训练的连接，从而实现端到端联合训练。此外，作为一个副产品，STN还能提供比原始裁剪器更精确的裁剪部分。由于这两个优点，我们的方法显著提高了原始模型的准确性。我们的模型在标准面部解析数据集海伦数据集上取得了有竞争力的性能。它在CelebAMask-HQ数据集上也取得了优异的性能，证明了其良好的泛化能力。我们的代码已在https://github.com/aod321/STN-iCNN上发布。
