Molecular and Neuroimaging Engineering Research Center of Ministry of Education, School of Life Science and Technology, Xidian University, Xi'an 710071, China.
Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV 26505, USA.
Sensors (Basel). 2022 Dec 5;22(23):9500. doi: 10.3390/s22239500.
Cross-spectral face verification between short-wave infrared (SWIR) and visible light (VIS) face images poses a challenge, which is motivated by various real-world applications such as surveillance at night time or in harsh environments. This paper proposes a hybrid solution that takes advantage of both traditional feature engineering and modern deep learning techniques to overcome the issue of as encountered in the SWIR band. Firstly, the paper revisits the theory of measurement levels. Then, two new operators are introduced which act at the nominal and interval levels of measurement and are named the Nominal Measurement Descriptor (NMD) and the Interval Measurement Descriptor (IMD), respectively. A composite operator Gabor Multiple-Level Measurement (GMLM) is further proposed which fuses multiple levels of measurement. Finally, the fused features of GMLM are passed through a succinct and efficient neural network based on PCA. The network selects informative features and also performs the recognition task. The overall framework is named GMLM-CNN. It is compared to both traditional hand-crafted operators as well as recent deep learning-based models that are state-of-the-art, in terms of cross-spectral verification performance. Experiments are conducted on a dataset which comprises frontal VIS and SWIR faces acquired at varying standoffs. Experimental results demonstrate that, in the presence of limited data, the proposed hybrid method GMLM-CNN outperforms all the other methods.
跨谱（短波长红外 (SWIR) 与可见光 (VIS)）人脸验证在各种实际应用中存在挑战，例如夜间或恶劣环境下的监控。本文提出了一种混合解决方案，结合传统特征工程和现代深度学习技术，克服了 SWIR 波段中出现的问题。首先，本文回顾了度量水平理论。然后，引入了两个新的算子，它们分别作用于标称和区间度量水平，分别命名为标称度量描述符（NMD）和区间度量描述符（IMD）。进一步提出了一种复合算子 Gabor 多水平测量（GMLM），它融合了多个度量水平。最后，将 GMLM 的融合特征通过基于 PCA 的简洁有效的神经网络传递。该网络选择信息丰富的特征并执行识别任务。整个框架命名为 GMLM-CNN。在跨谱验证性能方面，将其与传统的手工制作算子以及最新的基于深度学习的最先进模型进行了比较。实验在一个包含不同距离的正面 VIS 和 SWIR 人脸的数据集上进行。实验结果表明，在数据有限的情况下，所提出的混合方法 GMLM-CNN 优于所有其他方法。
