Delay constrained multiplexing of video streams using dual-frame video coding.

We consider the multiplexing problem of transmitting multiple video source streams from a server over a shared channel. We use dual-frame video coding with high-quality Long-Term Reference (LTR) frames and propose multiplexing methods to reduce the sum of mean squared error for all the video streams. This paper makes several improvements to dual-frame video coding. A simple motion activity detection algorithm is used to choose the location of LTR frames as well as the number of bits given to such frames. An adaptive buffer-constrained rate-control algorithm is devised to accommodate the extra bits of the high-quality LTR frames. Multiplexing of video streams is studied under the constraint of a video encoder delay buffer. Using H.264/AVC, the results show considerable improvement over baseline schemes such as H.264 rate control when the video streams are encoded individually and over multiplexing methods proposed previously in the literature. The high-quality LTR frames are offset in time among different video streams. This provides the benefits of dual-frame coding with high-quality LTR frames while still fitting under the constraint of an output delay buffer.