InnoFusion Technology, 3F-10, No. 5, Taiyuan 1st street, Zhubei, Hsinchu 30288, Taiwan.
Communication Engineering Department, National Central University, Taoyuan 320317, Taiwan.
Sensors (Basel). 2021 Mar 1;21(5):1682. doi: 10.3390/s21051682.
Rich information is provided by 360-degree videos. However, non-uniform geometric deformation caused by sphere-to-plane projection significantly decreases tracking accuracy of existing trackers, and the huge amount of data makes it difficult to achieve real-time tracking. Thus, this paper proposes a Siamese networks-based people tracker using template update for 360-degree equi-angular cubemap (EAC) format videos. Face stitching overcomes the problem of content discontinuity of the EAC format and avoids raising new geometric deformation in stitched images. Fully convolutional Siamese networks enable tracking at high speed. Mostly important, to be robust against combination of non-uniform geometric deformation of the EAC format and partial occlusions caused by zero padding in stitched images, this paper proposes a novel Bayes classifier-based timing detector of template update by referring to the linear discriminant feature and statistics of a score map generated by Siamese networks. Experimental results show that the proposed scheme significantly improves tracking accuracy of the fully convolutional Siamese networks SiamFC on the EAC format with operation beyond the frame acquisition rate. Moreover, the proposed score map-based timing detector of template update outperforms state-of-the-art score map-based timing detectors.
360 度视频提供了丰富的信息。然而，由于球体到平面的投影导致的非均匀几何变形，极大地降低了现有跟踪器的跟踪精度，并且大量的数据使得实时跟踪变得困难。因此，本文提出了一种基于 Siamese 网络的、使用模板更新的 360 度等角立方体贴图（EAC）格式视频的人员跟踪器。人脸拼接克服了 EAC 格式的内容不连续性问题，并避免了在拼接图像中产生新的几何变形。全卷积 Siamese 网络实现了高速跟踪。最重要的是，为了抵抗 EAC 格式的非均匀几何变形和拼接图像中零填充引起的部分遮挡的组合影响，本文提出了一种基于贝叶斯分类器的模板更新定时检测方法，该方法通过参考线性判别特征和 Siamese 网络生成的得分图的统计信息来实现。实验结果表明，与全卷积 Siamese 网络 SiamFC 相比，所提出的方案在超过帧率采集的 EAC 格式上显著提高了跟踪精度。此外，所提出的基于得分图的模板更新定时检测器优于最新的基于得分图的定时检测器。
