Feng Zhanxiang, Lai Jianhuang, Xie Xiaohua
IEEE Trans Image Process. 2019 Jul 17. doi: 10.1109/TIP.2019.2928126.
Traditional person re-identification (re-id) methods perform poorly under changing illuminations. This situation can be addressed by using dual-cameras that capture visible images in a bright environment and infrared images in a dark environment. Yet, this scheme needs to solve the visible-infrared matching issue, which is largely under-studied. Matching pedestrians across heterogeneous modalities is extremely challenging because of different visual characteristics. In this paper, we propose a novel framework that employ modality-specific networks to tackle with the heterogeneous matching problem. The proposed framework utilizes the modality-related information and extracts modality-specific representations (MSR) by constructing an individual network for each modality. In addition, a cross-modality Euclidean constraint is introduced to narrow the gap between different networks. We also integrate the modality-shared layers into modality-specific networks to extract shareable information and use a modality-shared identity loss to facilitate the extraction of modality-invariant features. Then a modality-specific discriminant metric is learned for each domain to strengthen the discriminative power of MSR. Eventually, we use a view classifier to learn view information. The experiments demonstrate that the MSR effectively improves the performance of deep networks on VI-REID and remarkably outperforms the state-of-the-art methods.
传统的行人重识别（re-id）方法在光照变化的情况下表现不佳。通过使用双摄像头可以解决这种情况，即在明亮环境中捕获可见光图像，在黑暗环境中捕获红外图像。然而，这种方案需要解决可见光-红外匹配问题，而这在很大程度上尚未得到充分研究。由于视觉特征不同，跨异构模态匹配行人极具挑战性。在本文中，我们提出了一种新颖的框架，该框架采用特定模态网络来解决异构匹配问题。所提出的框架利用模态相关信息，并通过为每个模态构建单独的网络来提取特定模态表示（MSR）。此外，引入了跨模态欧几里得约束以缩小不同网络之间的差距。我们还将模态共享层集成到特定模态网络中以提取可共享信息，并使用模态共享身份损失来促进模态不变特征的提取。然后为每个域学习特定模态判别度量以增强MSR的判别能力。最后，我们使用视图分类器来学习视图信息。实验表明，MSR有效地提高了深度网络在可见光-红外行人重识别（VI-REID）上的性能，并且显著优于现有方法。
