Vu Hoai Nam, Nguyen Mai Huong, Pham Cuong
Department of Computer Science, Posts and Telecommunications Institute of Technology, Hanoi, 12110 Vietnam.
Department of Computer Vision, Aimesoft., JSC, Hanoi, 11310 Vietnam.
Appl Intell (Dordr). 2022;52(5):5497-5512. doi: 10.1007/s10489-021-02728-1. Epub 2021 Aug 14.
Face recognition is one of the most common biometric authentication methods as its feasibility while convenient use. Recently, the COVID-19 pandemic is dramatically spreading throughout the world, which seriously leads to negative impacts on people's health and economy. Wearing masks in public settings is an effective way to prevent viruses from spreading. However, masked face recognition is a highly challenging task due to the lack of facial feature information. In this paper, we propose a method that takes advantage of the combination of deep learning and Local Binary Pattern (LBP) features to recognize the masked face by utilizing RetinaFace, a joint extra-supervised and self-supervised multi-task learning face detector that can deal with various scales of faces, as a fast yet effective encoder. In addition, we extract local binary pattern features from masked face's eye, forehead and eyebow areas and combine them with features learnt from RetinaFace into a unified framework for recognizing masked faces. In addition, we collected a dataset named COMASK20 from 300 subjects at our institution. In the experiment, we compared our proposed system with several state of the art face recognition methods on the published Essex dataset and our self-collected dataset COMASK20. With the recognition results of 87% f1-score on the COMASK20 dataset and 98% f1-score on the Essex dataset, these demonstrated that our proposed system outperforms Dlib and InsightFace, which has shown the effectiveness and suitability of the proposed method. The COMASK20 dataset is available on https://github.com/tuminguyen/COMASK20 for research purposes.
人脸识别是最常见的生物特征认证方法之一，因为它既可行又使用方便。最近，新冠疫情在全球急剧蔓延，严重影响了人们的健康和经济。在公共场所佩戴口罩是防止病毒传播的有效方法。然而，由于缺乏面部特征信息，戴口罩人脸识别是一项极具挑战性的任务。在本文中，我们提出了一种方法，利用深度学习和局部二值模式（LBP）特征的组合，通过使用RetinaFace（一种联合额外监督和自监督的多任务学习人脸检测器，可处理各种尺度的人脸）作为快速且有效的编码器来识别戴口罩的人脸。此外，我们从戴口罩人脸的眼睛、额头和眉毛区域提取局部二值模式特征，并将它们与从RetinaFace学到的特征结合到一个统一的框架中，用于识别戴口罩的人脸。此外，我们在本校从300名受试者那里收集了一个名为COMASK20的数据集。在实验中，我们将我们提出的系统与已发表的埃塞克斯数据集和我们自己收集的COMASK20数据集上的几种先进人脸识别方法进行了比较。在COMASK20数据集上的识别结果为f1分数87%，在埃塞克斯数据集上为f1分数98%，这些结果表明我们提出的系统优于Dlib和InsightFace，这证明了所提方法的有效性和适用性。COMASK20数据集可在https://github.com/tuminguyen/COMASK20上获取，供研究使用。
