Queen Mary University of London, UK.
IEEE Trans Pattern Anal Mach Intell. 2010 Nov;32(11):1940-54. doi: 10.1109/TPAMI.2010.50.
In this work, we propose a dynamic texture-based approach to the recognition of facial Action Units (AUs, atomic facial gestures) and their temporal models (i.e., sequences of temporal segments: neutral, onset, apex, and offset) in near-frontal-view face videos. Two approaches to modeling the dynamics and the appearance in the face region of an input video are compared: an extended version of Motion History Images and a novel method based on Nonrigid Registration using Free-Form Deformations (FFDs). The extracted motion representation is used to derive motion orientation histogram descriptors in both the spatial and temporal domain. Per AU, a combination of discriminative, frame-based GentleBoost ensemble learners and dynamic, generative Hidden Markov Models detects the presence of the AU in question and its temporal segments in an input image sequence. When tested for recognition of all 27 lower and upper face AUs, occurring alone or in combination in 264 sequences from the MMI facial expression database, the proposed method achieved an average event recognition accuracy of 89.2 percent for the MHI method and 94.3 percent for the FFD method. The generalization performance of the FFD method has been tested using the Cohn-Kanade database. Finally, we also explored the performance on spontaneous expressions in the Sensitive Artificial Listener data set.
在这项工作中，我们提出了一种基于动态纹理的方法，用于识别近正面人脸视频中的面部动作单元（AU，原子面部表情）及其时间模型（即时间片段序列：中性、起始、顶点和结束）。比较了两种用于建模输入视频中面部区域动态和外观的方法：运动历史图像的扩展版本和基于自由变形（FFD）的非刚性配准的新方法。提取的运动表示用于在空间和时间域中得出运动方向直方图描述符。对于每个 AU，基于判别、基于帧的 GentleBoost 集成学习器和动态、生成性隐马尔可夫模型的组合检测输入图像序列中存在的 AU 及其时间片段。当针对 MMI 面部表情数据库中的 264 个序列中的 27 个上下脸 AU 进行识别时，该方法使用 MHI 方法的平均事件识别准确率为 89.2%，使用 FFD 方法的平均事件识别准确率为 94.3%。还使用 Cohn-Kanade 数据库测试了 FFD 方法的泛化性能。最后，我们还在敏感人工听众数据集上探索了对自发表情的性能。
