Lu Andong, Qian Cun, Li Chenglong, Tang Jin, Wang Liang
IEEE Trans Neural Netw Learn Syst. 2025 Mar;36(3):4118-4131. doi: 10.1109/TNNLS.2022.3157594. Epub 2025 Feb 28.
Low-quality modalities contain not only a lot of noisy information but also some discriminative features in RGB-Thermal (RGBT) tracking. However, the potentials of low-quality modalities are not well explored in existing RGBT tracking algorithms. In this work, we propose a novel duality-gated mutual condition network to fully exploit the discriminative information of all modalities while suppressing the effects of data noise. In specific, we design a mutual condition module, which takes the discriminative information of a modality as the condition to guide feature learning of target appearance in another modality. Such a module can effectively enhance target representations of all modalities even in the presence of low-quality modalities. To improve the quality of conditions and further reduce data noise, we propose a duality-gated mechanism and integrate it into the mutual condition module. To deal with the tracking failure caused by sudden camera motion, which often occurs in RGBT tracking, we design a resampling strategy based on optical flow. It does not increase much computational cost since we perform optical flow calculation only when the model prediction is unreliable and then execute resampling when the sudden camera motion is detected. Extensive experiments on four RGBT tracking benchmark datasets show that our method performs favorably against the state-of-the-art tracking algorithms.
在RGB-Thermal(RGBT)跟踪中,低质量模态不仅包含大量噪声信息,还包含一些判别性特征。然而,现有RGBT跟踪算法并未充分挖掘低质量模态的潜力。在这项工作中,我们提出了一种新颖的对偶门控互条件网络,以充分利用所有模态的判别信息,同时抑制数据噪声的影响。具体来说,我们设计了一个互条件模块,该模块将一种模态的判别信息作为条件,来指导另一模态中目标外观的特征学习。即使存在低质量模态,这样的模块也能有效地增强所有模态的目标表示。为了提高条件的质量并进一步降低数据噪声,我们提出了一种对偶门控机制,并将其集成到互条件模块中。为了处理RGBT跟踪中经常出现的由相机突然运动导致的跟踪失败问题,我们设计了一种基于光流的重采样策略。由于我们仅在模型预测不可靠时进行光流计算,然后在检测到相机突然运动时执行重采样,因此该策略不会增加太多计算成本。在四个RGBT跟踪基准数据集上进行的大量实验表明,我们的方法优于当前的先进跟踪算法。