IEEE Trans Vis Comput Graph. 2017 Dec;23(12):2586-2598. doi: 10.1109/TVCG.2016.2641442. Epub 2016 Dec 19.
This paper proposes a real-time method for 3D eye performance reconstruction using a single RGBD sensor. Combined with facial surface tracking, our method generates more pleasing facial performance with vivid eye motions. In our method, a novel scheme is proposed to estimate eyeball motions by minimizing the differences between a rendered eyeball and the recorded image. Our method considers and handles different appearances of human irises, lighting variations and highlights on images via the proposed eyeball model and the -based optimization. Robustness and real-time optimization are achieved through the novel 3D Taylor expansion-based linearization. Furthermore, we propose an online bidirectional regression method to handle occlusions and other tracking failures on either of the two eyes from the information of the opposite eye. Experiments demonstrate that our technique achieves robust and accurate eye performance reconstruction for different iris appearances, with various head/face/eye motions, and under different lighting conditions.
本文提出了一种使用单个 RGBD 传感器进行 3D 眼部性能重建的实时方法。结合面部表面跟踪，我们的方法通过生成更生动的眼部运动来生成更令人愉悦的面部表情。在我们的方法中，提出了一种通过最小化渲染眼球和记录图像之间的差异来估计眼球运动的新方案。我们的方法通过提出的眼球模型和基于的优化来考虑和处理人虹膜的不同外观、图像中的光照变化和高光。通过新颖的基于 3D Taylor 展开的线性化实现了稳健性和实时优化。此外，我们提出了一种在线双向回归方法，以便从另一只眼睛的信息处理两只眼睛中的任何一只的遮挡和其他跟踪失败。实验表明，我们的技术在不同的虹膜外观、各种头部/面部/眼睛运动以及不同的光照条件下，都能实现稳健、准确的眼部性能重建。
