IEEE Trans Image Process. 2014 Jan;23(1):399-412. doi: 10.1109/TIP.2013.2288139.
This paper presents a new framework for motion compensated frame rate up conversion (FRUC) based on variational image fusion. The proposed algorithm consists of two steps: 1) generation of multiple intermediate interpolated frames and 2) fusion of those intermediate frames. In the first step, we determine four different sets of the motion vector field using four neighboring frames. We then generate intermediate interpolated frames corresponding to the determined four sets of the motion vector field, respectively. Multiple sets of the motion vector field are used to solve the occlusion problem in motion estimation. In the second step, the four intermediate interpolated frames are fused into a single frame via a variational image fusion process. For effective fusion, we determine fusion weights for each intermediate interpolated frame by minimizing the energy, which consists of a weighted-L1-norm based data energy and gradient-driven smoothness energy. Experimental results demonstrate that the proposed algorithm improves the performance of FRUC compared with the existing algorithms.
本文提出了一种基于变分图像融合的运动补偿帧率上转换 (FRUC) 的新框架。所提出的算法由两个步骤组成：1）生成多个中间插值帧；2）融合这些中间帧。在第一步中，我们使用四个相邻的帧来确定四个不同的运动矢量场集合。然后，我们分别为所确定的四个运动矢量场集合生成中间插值帧。使用多组运动矢量场来解决运动估计中的遮挡问题。在第二步中，通过变分图像融合过程将四个中间插值帧融合为单个帧。为了实现有效的融合，我们通过最小化能量来确定每个中间插值帧的融合权重，该能量由基于加权 L1 范数的数据能量和梯度驱动的平滑度能量组成。实验结果表明，与现有算法相比，所提出的算法提高了 FRUC 的性能。
