IEEE Trans Image Process. 2018 Feb;27(2):678-691. doi: 10.1109/TIP.2017.2767782.
Frame rate up conversion (FRUC) can improve the visual quality by interpolating new intermediate frames. However, high frame rate videos by FRUC are confronted with more bitrate consumption or annoying artifacts of interpolated frames. In this paper, a novel integration framework of FRUC and high efficiency video coding (HEVC) is proposed based on rate-distortion optimization, and the interpolated frames can be reconstructed at encoder side with low bitrate cost and high visual quality. First, joint motion estimation (JME) algorithm is proposed to obtain robust motion vectors, which are shared between FRUC and video coding. What's more, JME is embedded into the coding loop and employs the original motion search strategy in HEVC coding. Then, the frame interpolation is formulated as a rate-distortion optimization problem, where both the coding bitrate consumption and visual quality are taken into account. Due to the absence of original frames, the distortion model for interpolated frames is established according to the motion vector reliability and coding quantization error. Experimental results demonstrate that the proposed framework can achieve 21% ~ 42% reduction in BDBR, when compared with the traditional methods of FRUC cascaded with coding.
帧率上转换 (FRUC) 可以通过插值新的中间帧来提高视觉质量。然而，通过 FRUC 生成的高帧率视频面临着更高的比特率消耗或插值帧的恼人伪像问题。在本文中，我们提出了一种基于率失真优化的 FRUC 和高效视频编码 (HEVC) 的新型集成框架，通过低比特率成本和高视觉质量，在编码器端可以重建插值帧。首先，提出了联合运动估计 (JME) 算法，以获得稳健的运动向量，这些运动向量在 FRUC 和视频编码之间共享。此外，JME 被嵌入到编码循环中，并采用 HEVC 编码中的原始运动搜索策略。然后，将帧插值表示为一个率失真优化问题，同时考虑了编码比特率消耗和视觉质量。由于没有原始帧，因此根据运动向量可靠性和编码量化误差来建立插值帧的失真模型。实验结果表明，与传统的 FRUC 与编码级联方法相比，所提出的框架可以将 BDBR 降低 21%至 42%。
