Toward based on concentrated multi-scale linear attention real-time UAV tracking using joint natural language specification.

Based on natural language specification unmanned aerial vehicle tracking mission goal is to automatically and continuously track the target in subsequent frames by natural language descriptions. Existing tracking methods typically handle this problem through two separate steps: visual grounding and object tracking. However, this independent solution would result in ignoring the relationship between visual grounding and object tracking, e.g., natural language can provide semantic information about the target, and this solution would also result in an inability to train end-to-end. Therefore, we propose a framework based on natural language specification that integrates visual grounding and object tracking, redefining it as a unified task. This framework can track the object based on a given natural language reference. First, the proposed triangular integration effectively establishes the relationship between natural language and images (template image and search image). Then in order to accomplish multi-scale learning and global receptive field, and effectively improve the flexibility of the method to the visual characteristics of the tracking target, we designed a new lightweight concentrated multi-scale linear attention. Additionally, to reduce computational complexity, we introduced residuals. Finally, experiments conducted on six UAV tracking datasets showed that our tracker achieved accuracy, success rate, and average speed of 0.819, 0.654, and 61 FPS, respectively, outperforming other state-of-the-art trackers.