IEEE Trans Pattern Anal Mach Intell. 2019 Jul;41(7):1761-1773. doi: 10.1109/TPAMI.2018.2842770. Epub 2018 Jun 1.
Heterogeneous face recognition (HFR) aims at matching facial images acquired from different sensing modalities with mission-critical applications in forensics, security and commercial sectors. However, HFR presents more challenging issues than traditional face recognition because of the large intra-class variation among heterogeneous face images and the limited availability of training samples of cross-modality face image pairs. This paper proposes the novel Wasserstein convolutional neural network (WCNN) approach for learning invariant features between near-infrared (NIR) and visual (VIS) face images (i.e., NIR-VIS face recognition). The low-level layers of the WCNN are trained with widely available face images in the VIS spectrum, and the high-level layer is divided into three parts: the NIR layer, the VIS layer and the NIR-VIS shared layer. The first two layers aim at learning modality-specific features, and the NIR-VIS shared layer is designed to learn a modality-invariant feature subspace. The Wasserstein distance is introduced into the NIR-VIS shared layer to measure the dissimilarity between heterogeneous feature distributions. W-CNN learning is performed to minimize the Wasserstein distance between the NIR distribution and the VIS distribution for invariant deep feature representations of heterogeneous face images. To avoid the over-fitting problem on small-scale heterogeneous face data, a correlation prior is introduced on the fully-connected WCNN layers to reduce the size of the parameter space. This prior is implemented by a low-rank constraint in an end-to-end network. The joint formulation leads to an alternating minimization for deep feature representation at the training stage and an efficient computation for heterogeneous data at the testing stage. Extensive experiments using three challenging NIR-VIS face recognition databases demonstrate the superiority of the WCNN method over state-of-the-art methods.
异质人脸识别（HFR）旨在将不同感测模式获取的面部图像与取证、安全和商业领域的关键任务应用程序进行匹配。然而，与传统的人脸识别相比，HFR 面临更多的挑战，因为异质人脸图像之间存在较大的类内变化，并且跨模态人脸图像对的训练样本可用性有限。本文提出了一种新颖的 Wasserstein 卷积神经网络（WCNN）方法，用于学习近红外（NIR）和可见光（VIS）人脸图像（即 NIR-VIS 人脸识别）之间的不变特征。WCNN 的低层使用 VIS 光谱中广泛可用的人脸图像进行训练，高层分为三部分：NIR 层、VIS 层和 NIR-VIS 共享层。前两层旨在学习特定于模态的特征，而 NIR-VIS 共享层旨在学习模态不变的特征子空间。将 Wasserstein 距离引入到 NIR-VIS 共享层中，以测量异质特征分布之间的差异。W-CNN 学习旨在最小化 NIR 分布和 VIS 分布之间的 Wasserstein 距离，以获得异质人脸图像的不变深度特征表示。为了避免在小尺度异质人脸数据上的过拟合问题，在全连接 WCNN 层上引入相关先验，以减小参数空间的大小。该先验通过端到端网络中的低秩约束来实现。联合公式在训练阶段导致深度特征表示的交替最小化，并在测试阶段实现高效的异质数据计算。使用三个具有挑战性的 NIR-VIS 人脸识别数据库进行的广泛实验表明，WCNN 方法优于最先进的方法。
