National Institute of Informatics, Tokyo 101-8430, Japan.
IEEE Trans Image Process. 2011 Mar;20(3):744-61. doi: 10.1109/TIP.2010.2070074. Epub 2010 Aug 26.
While much of multiview video coding focuses on the rate-distortion performance of compressing all frames of all views for storage or non-interactive video delivery over networks, we address the problem of designing a frame structure to enable interactive multiview streaming, where clients can interactively switch views during video playback. Thus, as a client is playing back successive frames (in time) for a given view, it can send a request to the server to switch to a different view while continuing uninterrupted temporal playback. Noting that standard tools for random access (i.e., I-frame insertion) can be bandwidth-inefficient for this application, we propose a redundant representation of I-, P-, and "merge" frames, where each original picture can be encoded into multiple versions, appropriately trading off expected transmission rate with storage, to facilitate view switching. We first present ad hoc frame structures with good performance when the view-switching probabilities are either very large or very small. We then present optimization algorithms that generate more general frame structures with better overall performance for the general case. We show in our experiments that we can generate redundant frame structures offering a range of tradeoff points between transmission and storage, e.g., outperforming simple I-frame insertion structures by up to 45% in terms of bandwidth efficiency at twice the storage cost.
虽然多视点视频编码的大部分工作都集中在压缩所有视点的所有帧以进行存储或通过网络进行非交互式视频传输的率失真性能上，但我们解决了设计帧结构以实现交互式多视点流的问题，在这种流中，客户端可以在视频播放期间交互式地切换视点。因此，当客户端按时间顺序播放给定视点的连续帧时，它可以向服务器发送请求，在继续进行时间播放的同时切换到不同的视点。请注意，对于这种应用，标准的随机访问工具（即 I 帧插入）可能会导致带宽效率低下，因此我们提出了 I 帧、P 帧和“合并”帧的冗余表示，其中每个原始图像可以编码成多个版本，适当权衡预期传输速率和存储，以方便视点切换。我们首先提出了在视点切换概率非常大或非常小的情况下具有良好性能的特定帧结构。然后，我们提出了生成具有更一般总体性能的更一般帧结构的优化算法。我们在实验中表明，我们可以生成具有在传输和存储之间提供一系列权衡点的冗余帧结构，例如，在带宽效率方面，与简单的 I 帧插入结构相比，存储成本增加一倍时，性能提高高达 45%。
