College of Information Science and Engineering, Huaqiao University, Xiamen 361021, China.
College of Engineering, Huaqiao University, Quanzhou 362021, China.
Sensors (Basel). 2023 Jan 27;23(3):1426. doi: 10.3390/s23031426.
Visible-infrared person re-identification (VIPR) has great potential for intelligent transportation systems for constructing smart cities, but it is challenging to utilize due to the huge modal discrepancy between visible and infrared images. Although visible and infrared data can appear to be two domains, VIPR is not identical to domain adaptation as it can massively eliminate modal discrepancies. Because VIPR has complete identity information on both visible and infrared modalities, once the domain adaption is overemphasized, the discriminative appearance information on the visible and infrared domains would drain. For that, we propose a novel margin-based modal adaptive learning (MMAL) method for VIPR in this paper. On each domain, we apply triplet and label smoothing cross-entropy functions to learn appearance-discriminative features. Between the two domains, we design a simple yet effective marginal maximum mean discrepancy (M3D) loss function to avoid an excessive suppression of modal discrepancies to protect the features' discriminative ability on each domain. As a result, our MMAL method could learn modal-invariant yet appearance-discriminative features for improving VIPR. The experimental results show that our MMAL method acquires state-of-the-art VIPR performance, e.g., on the RegDB dataset in the visible-to-infrared retrieval mode, the rank-1 accuracy is 93.24% and the mean average precision is 83.77%.
可见光-近红外人像再识别（VIPR）在构建智慧城市的智能交通系统中有很大的应用潜力，但由于可见光和红外图像之间存在巨大的模态差异，因此利用起来具有挑战性。虽然可见光和红外数据看起来像是两个不同的领域，但 VIPR 与域自适应不同，因为它可以极大地消除模态差异。由于 VIPR 在可见光和红外两种模态上都具有完整的身份信息，一旦过分强调域自适应，那么在可见光和红外域上的判别外观信息就会流失。为此，我们在本文中提出了一种新颖的基于边缘的模态自适应学习（MMAL）方法用于 VIPR。在每个模态上，我们应用三元组和标签平滑交叉熵函数来学习外观判别特征。在两个模态之间，我们设计了一个简单而有效的边缘最大均值差异（M3D）损失函数，以避免过度抑制模态差异，从而保护每个模态上特征的判别能力。因此，我们的 MMAL 方法可以学习模态不变但外观判别特征，从而提高 VIPR 的性能。实验结果表明，我们的 MMAL 方法在可见光到红外检索模式下的 RegDB 数据集上取得了最先进的 VIPR 性能，例如，排名第一的准确率为 93.24%，平均准确率为 83.77%。
