Peng Hao, Zhang Yun, Zhang Fang-Lue
School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand.
School of Media Engineering, Zhejiang Key Laboratory of Film and TV Media Technology, Communication University of Zhejiang, Hangzhou, People's Republic of China.
J R Soc N Z. 2025 Jun 22;55(6):2169-2197. doi: 10.1080/03036758.2025.2519148. eCollection 2025.
Recently, 360-degree visual tracking has become increasingly important in 360-degree video processing technology. Although visual tracking technology in 2D videos has gradually matured, there is no universal method for visual tracking in 360-degree videos that can effectively address image stretching and object deformation caused by the equirectangular representation of 360-degree images. In this paper, we propose a two-part method for 360-degree visual tracking. The first part is a general method that can be integrated into any 2D visual tracking system to be applied to 360-degree videos. This part converts equirectangular images into 2D gnomonic projections, enabling the use of existing 2D tracking algorithms while mitigating image distortion. Then, building upon the UPDT algorithm, the second part integrates the general 360-degree visual tracking method with additional enhancements to improve robustness and efficiency in 360-degree visual tracking. Furthermore, when tracking performance deteriorates, it combines results from the sample set and trajectory prediction to achieve more robust and accurate tracking. In our experiments, We use two datasets in 360-degree equirectangular representation to demonstrate the effectiveness and advantages of our proposed method. Additionally, we explore the application of 360-degree visual tracking methods in editing, enabling the automatic manipulation of moving objects in 360-degree videos.
近年来，360度视觉跟踪在360度视频处理技术中变得越来越重要。尽管二维视频中的视觉跟踪技术已逐渐成熟，但在360度视频的视觉跟踪方面，尚无一种通用方法能够有效解决由360度图像的等矩形表示所引起的图像拉伸和物体变形问题。在本文中，我们提出了一种用于360度视觉跟踪的两部分方法。第一部分是一种通用方法，可集成到任何二维视觉跟踪系统中以应用于360度视频。这部分将等矩形图像转换为二维球极平面投影，从而能够使用现有的二维跟踪算法，同时减轻图像失真。然后，在UPDT算法的基础上，第二部分将通用的360度视觉跟踪方法与额外的增强功能相结合，以提高360度视觉跟踪的鲁棒性和效率。此外，当跟踪性能下降时，它会结合样本集的结果和轨迹预测，以实现更稳健、准确的跟踪。在我们的实验中，我们使用两个360度等矩形表示的数据集来证明我们所提出方法的有效性和优势。此外，我们还探索了360度视觉跟踪方法在编辑方面的应用，实现对360度视频中移动物体的自动操作。
