IEEE Trans Image Process. 2014 Apr;23(4):1615-28. doi: 10.1109/TIP.2014.2305843.
Video retargeting is a useful technique to adapt a video to a desired display resolution. It aims to preserve the information contained in the original video and the shapes of salient objects while maintaining the temporal coherence of contents in the video. Existing video retargeting schemes achieve temporal coherence via constraining each region/pixel to be deformed consistently with its corresponding region/pixel in neighboring frames. However, these methods often distort the shapes of salient objects, since they do not ensure the content consistency for regions/pixels constrained to be coherently deformed along time axis. In this paper, we propose a video retargeting scheme to simultaneously meet the two requirements. Our method first segments a video clip into spatiotemporal grids called grid flows, where the consistency of the content associated with a grid flow is maintained while retargeting the grid flow. After that, due to the coarse granularity of grid, there still may exist content inconsistency in some grid flows. We exploit the temporal redundancy in a grid flow to avoid that the grids with inconsistent content be incorrectly constrained to be coherently deformed. In particular, we use grid flows to select a set of key-frames which summarize a video clip, and resize subgrid-flows in these key-frames. We then resize the remaining nonkey-frames by simply interpolating their grid contents from the two nearest retargeted key-frames. With the key-frame-based scheme, we only need to solve a small-scale quadratic programming problem to resize subgrid-flows and perform grid interpolation, leading to low computation and memory costs. The experimental results demonstrate the superior performance of our scheme.
视频重定向是一种将视频适配到期望显示分辨率的有用技术。它旨在保留原始视频中包含的信息和显著对象的形状，同时保持视频内容的时间连贯性。现有的视频重定向方案通过约束每个区域/像素与相邻帧中的对应区域/像素一致变形来实现时间连贯性。然而，这些方法经常会扭曲显著对象的形状，因为它们不能确保沿时间轴一致变形的区域/像素的内容一致性。在本文中，我们提出了一种同时满足这两个要求的视频重定向方案。我们的方法首先将视频剪辑分割成称为网格流的时空网格，在重定向网格流的同时保持与网格流相关的内容的一致性。之后，由于网格的粗粒度，在某些网格流中仍然可能存在内容不一致。我们利用网格流中的时间冗余来避免将具有不一致内容的网格错误地约束为一致变形。具体来说，我们使用网格流来选择一组总结视频剪辑的关键帧，并调整这些关键帧中的子网格流。然后，我们通过简单地从最近的两个重定向关键帧中插值其网格内容来调整其余非关键帧的大小。通过基于关键帧的方案，我们只需要解决一个小规模的二次规划问题来调整子网格流和执行网格插值，从而降低计算和存储成本。实验结果表明了我们方案的优越性能。
