Tang Peng, Wang Chunyu, Wang Xinggang, Liu Wenyu, Zeng Wenjun, Wang Jingdong
IEEE Trans Pattern Anal Mach Intell. 2020 May;42(5):1272-1278. doi: 10.1109/TPAMI.2019.2910529. Epub 2019 Apr 11.
Compared with object detection in static images, object detection in videos is more challenging due to degraded image qualities. An effective way to address this problem is to exploit temporal contexts by linking the same object across video to form tubelets and aggregating classification scores in the tubelets. In this paper, we focus on obtaining high quality object linking results for better classification. Unlike previous methods that link objects by checking boxes between neighboring frames, we propose to link in the same frame. To achieve this goal, we extend prior methods in following aspects: (1) a cuboid proposal network that extracts spatio-temporal candidate cuboids which bound the movement of objects; (2) a short tubelet detection network that detects short tubelets in short video segments; (3) a short tubelet linking algorithm that links temporally-overlapping short tubelets to form long tubelets. Experiments on the ImageNet VID dataset show that our method outperforms both the static image detector and the previous state of the art. In particular, our method improves results by 8.8 percent over the static image detector for fast moving objects.
与静态图像中的目标检测相比，视频中的目标检测由于图像质量下降而更具挑战性。解决此问题的有效方法是通过跨视频链接同一目标以形成小目标管并汇总小目标管中的分类分数来利用时间上下文。在本文中，我们专注于获得高质量的目标链接结果以进行更好的分类。与之前通过检查相邻帧之间的框来链接目标的方法不同，我们建议在同一帧中进行链接。为实现这一目标，我们在以下方面扩展了先前的方法：（1）一个长方体提议网络，用于提取界定目标运动的时空候选长方体；（2）一个短目标管检测网络，用于在短视频片段中检测短目标管；（3）一种短目标管链接算法，用于链接时间上重叠的短目标管以形成长目标管。在ImageNet VID数据集上的实验表明，我们的方法优于静态图像检测器和先前的现有技术。特别是，对于快速移动的目标，我们的方法比静态图像检测器的结果提高了8.8%。
