Cioppa Anthony, Braham Marc, Van Droogenbroeck Marc
Montefiore Institute, University of Liège, Quartier Polytech 1, Allée de la Découverte 10, 4000 Liège, Belgium.
J Imaging. 2020 Jun 18;6(6):50. doi: 10.3390/jimaging6060050.
The method of Semantic Background Subtraction (SBS), which combines semantic segmentation and background subtraction, has recently emerged for the task of segmenting moving objects in video sequences. While SBS has been shown to improve background subtraction, a major difficulty is that it combines two streams generated at different frame rates. This results in SBS operating at the slowest frame rate of the two streams, usually being the one of the semantic segmentation algorithm. We present a method, referred to as "Asynchronous Semantic Background Subtraction" (ASBS), able to combine a semantic segmentation algorithm with any background subtraction algorithm asynchronously. It achieves performances close to that of SBS while operating at the fastest possible frame rate, being the one of the background subtraction algorithm. Our method consists in analyzing the temporal evolution of pixel features to possibly replicate the decisions previously enforced by semantics when no semantic information is computed. We showcase ASBS with several background subtraction algorithms and also add a feedback mechanism that feeds the background model of the background subtraction algorithm to upgrade its updating strategy and, consequently, enhance the decision. Experiments show that we systematically improve the performance, even when the semantic stream has a much slower frame rate than the frame rate of the background subtraction algorithm. In addition, we establish that, with the help of ASBS, a real-time background subtraction algorithm, such as ViBe, stays real time and competes with some of the best non-real-time unsupervised background subtraction algorithms such as SuBSENSE.
语义背景减法（SBS）方法结合了语义分割和背景减法，最近已出现用于视频序列中移动物体的分割任务。虽然SBS已被证明能改进背景减法，但一个主要困难是它结合了以不同帧率生成的两个流。这导致SBS以两个流中最慢的帧率运行，通常是语义分割算法的帧率。我们提出一种称为“异步语义背景减法”（ASBS）的方法，能够将语义分割算法与任何背景减法算法异步结合。它在以背景减法算法的最快可能帧率运行时，实现了接近SBS的性能。我们的方法包括分析像素特征的时间演变，以便在未计算语义信息时可能复制先前由语义执行的决策。我们用几种背景减法算法展示了ASBS，还添加了一种反馈机制，该机制为背景减法算法的背景模型提供信息，以升级其更新策略，从而增强决策。实验表明，即使语义流的帧率比背景减法算法的帧率慢得多，我们也能系统地提高性能。此外，我们确定，在ASBS的帮助下，一种实时背景减法算法，如ViBe，能保持实时性，并能与一些最佳的非实时无监督背景减法算法，如SuBSENSE竞争。
