Annu Int Conf IEEE Eng Med Biol Soc. 2022 Jul;2022:2215-2218. doi: 10.1109/EMBC48229.2022.9871549.
Video motion magnification methods are motion visualization techniques that aim to magnify subtle and imper-ceptibly small motions in videos. They fall into two main groups where Eulerian methods work on the pixel grid with implicit motion information and Lagrangian methods use explicitly estimated motion and modify point trajectories. The motion in high framerate videos of faces can contain a wide variety of information that ranges from microexpressions over pulse or respiratory rate to cues on speech and affective state. In his work, we propose a novel strategy for Lagrangian motion magnification that integrates landmark information from the face as well as an approach to decompose facial motions in an unsupervised manner using sparse PCA. We decompose the estimated displacements into different movement components that are subsequently amplified selectively. We propose two approaches: A landmark-based decomposition into global and local movements and a decomposition into multiple coherent motion components based on sparse PCA. Optical flow estimation is performed using a state-of-the-art deep learning-based method that we retrain on a microexpression database. Clinical relevance- This method could be applied to the annotation and analysis of micromovements for neurocognitive assessment and even novel, medical applications where micro-motions of the face might play a role.
视频运动放大方法是一种运动可视化技术，旨在放大视频中细微和难以察觉的运动。它们分为两类，其中欧拉方法在像素网格上工作，具有隐含的运动信息，而拉格朗日方法则使用显式估计的运动并修改点轨迹。人脸高帧率视频中的运动可以包含各种信息，从微表情到脉搏或呼吸频率，再到语音和情感状态的线索。在他的工作中，我们提出了一种新的拉格朗日运动放大策略，该策略集成了面部的地标信息，以及一种使用稀疏 PCA 以无监督方式分解面部运动的方法。我们将估计的位移分解为不同的运动成分，然后有选择地放大这些成分。我们提出了两种方法：一种基于地标将运动分解为全局运动和局部运动，另一种基于稀疏 PCA 将运动分解为多个相干运动成分。光流估计使用一种基于深度学习的最新方法，我们在微表情数据库上对其进行了重新训练。临床相关性——该方法可应用于神经认知评估的微运动的注释和分析，甚至可以应用于面部微运动可能发挥作用的新的医学应用。
