Nagamatsu Genki, Nowara Ewa Magdalena, Pai Amruta, Veeraraghavan Ashok, Kawasaki Hiroshi
Annu Int Conf IEEE Eng Med Biol Soc. 2020 Jul;2020:1194-1197. doi: 10.1109/EMBC44109.2020.9176065.
Over the last few years, camera-based estimation of vital signs referred to as imaging photoplethysmography (iPPG) has garnered significant attention due to the relative simplicity, ease, unobtrusiveness and flexibility offered by such measurements. It is expected that iPPG may be integrated into a host of emerging applications in areas as diverse as autonomous cars, neonatal monitoring, and telemedicine. In spite of this potential, the primary challenge of non-contact camera-based measurements is the relative motion between the camera and the subjects. Current techniques employ 2D feature tracking to reduce the effect of subject and camera motion but they are limited to handling translational and in-plane motion. In this paper, we study, for the first-time, the utility of 3D face tracking to allow iPPG to retain robust performance even in presence of out-of-plane and large relative motions. We use a RGB-D camera to obtain 3D information from the subjects and use the spatial and depth information to fit a 3D face model and track the model over the video frames. This allows us to estimate correspondence over the entire video with pixel-level accuracy, even in the presence of out-of-plane or large motions. We then estimate iPPG from the warped video data that ensures per-pixel correspondence over the entire window-length used for estimation. Our experiments demonstrate improvement in robustness when head motion is large.
在过去几年中，基于摄像头的生命体征估计技术，即成像光电容积脉搏波描记法（iPPG），因其测量相对简单、便捷、不引人注意且灵活，受到了广泛关注。预计iPPG可集成到众多新兴应用中，涵盖自动驾驶汽车、新生儿监测和远程医疗等不同领域。尽管有此潜力，但基于非接触式摄像头测量的主要挑战是摄像头与被试者之间的相对运动。当前技术采用二维特征跟踪来减少被试者和摄像头运动的影响，但仅限于处理平移和平面内运动。在本文中，我们首次研究了三维面部跟踪的效用，以使iPPG即使在存在平面外和较大相对运动的情况下仍能保持稳健性能。我们使用RGB-D摄像头从被试者获取三维信息，并利用空间和深度信息拟合三维面部模型，并在视频帧上跟踪该模型。这使我们能够在整个视频中以像素级精度估计对应关系，即使存在平面外或大运动。然后，我们从扭曲的视频数据中估计iPPG，这确保了在用于估计的整个窗口长度上的逐像素对应关系。我们的实验表明，当头部运动较大时，稳健性有所提高。
