Instituto de Informática, Universidade Federal do Rio Grande do Sul, Porto Alegre, RS, Brazil.
IEEE Trans Image Process. 2010 Apr;19(4):1036-49. doi: 10.1109/TIP.2009.2038778. Epub 2009 Dec 18.
This work describes an approach for object-oriented video segmentation based on motion coherence. Using a tracking process based on adaptively sampled points (namely, particles), 2-D motion patterns are identified with an ensemble clustering approach. Particles are clustered to obtain a pixel-wise segmentation in space and time domains. The segmentation result is mapped to an image spatio-temporal feature space. Thus, the different constituent parts of the scene that move coherently along the video sequence are mapped to volumes in this spatio-temporal space. These volumes make the redundancy in the temporal sense more explicit, leading to potential gains in video coding applications. The proposed solution is robust and more generic than similar approaches for 2-D video segmentation found in the literature. In order to illustrate the potential advantages of using the proposed motion segmentation approach in video coding applications, the PSNR of the temporal predictions and the entropies of prediction errors obtained in our experiments are presented, and compared with other methods. Our experiments with real and synthetic sequences suggest that our method also could be used in other image processing and computer vision tasks, besides video coding, such as video information retrieval and video understanding.
这项工作描述了一种基于运动一致性的面向对象的视频分割方法。使用基于自适应采样点（即粒子）的跟踪过程，通过集成聚类方法识别 2D 运动模式。将粒子聚类以获得空间和时域的像素级分割。分割结果映射到图像时空特征空间。因此，沿着视频序列以一致性方式移动的场景的不同组成部分被映射到该时空空间中的体积。这些体积使时间上的冗余更加明显，从而在视频编码应用中获得潜在的收益。与文献中发现的类似 2D 视频分割方法相比，所提出的解决方案更健壮且更通用。为了说明在视频编码应用中使用所提出的运动分割方法的潜在优势，本文呈现了在实验中获得的时间预测的 PSNR 和预测误差的熵，并与其他方法进行了比较。我们使用真实和合成序列的实验表明，除了视频编码之外，我们的方法还可以用于图像处理和计算机视觉任务，例如视频信息检索和视频理解。
