IEEE Trans Pattern Anal Mach Intell. 2022 Sep;44(9):5826-5846. doi: 10.1109/TPAMI.2021.3067464. Epub 2022 Aug 4.
Unlike the conventional facial expressions, micro-expressions are involuntary and transient facial expressions capable of revealing the genuine emotions that people attempt to hide. Therefore, they can provide important information in a broad range of applications such as lie detection, criminal detection, etc. Since micro-expressions are transient and of low intensity, however, their detection and recognition is difficult and relies heavily on expert experiences. Due to its intrinsic particularity and complexity, video-based micro-expression analysis is attractive but challenging, and has recently become an active area of research. Although there have been numerous developments in this area, thus far there has been no comprehensive survey that provides researchers with a systematic overview of these developments with a unified evaluation. Accordingly, in this survey paper, we first highlight the key differences between macro- and micro-expressions, then use these differences to guide our research survey of video-based micro-expression analysis in a cascaded structure, encompassing the neuropsychological basis, datasets, features, spotting algorithms, recognition algorithms, applications and evaluation of state-of-the-art approaches. For each aspect, the basic techniques, advanced developments and major challenges are addressed and discussed. Furthermore, after considering the limitations of existing micro-expression datasets, we present and release a new dataset - called micro-and-macro expression warehouse (MMEW) - containing more video samples and more labeled emotion types. We then perform a unified comparison of representative methods on CAS(ME) for spotting, and on MMEW and SAMM for recognition, respectively. Finally, some potential future research directions are explored and outlined.
与传统面部表情不同，微表情是一种不由自主且短暂的面部表情，能够揭示人们试图隐藏的真实情感。因此，它们可以在广泛的应用中提供重要信息，例如测谎、犯罪检测等。然而，由于微表情是短暂且强度较低的，因此它们的检测和识别非常困难，并且严重依赖专家经验。由于其内在的特殊性和复杂性，基于视频的微表情分析具有吸引力但也具有挑战性，最近已成为一个活跃的研究领域。尽管在这一领域已经取得了许多进展，但迄今为止，还没有一篇全面的综述论文为研究人员提供了一个系统的概述，并对这些进展进行了统一的评估。因此，在本调查论文中，我们首先强调了宏观表情和微表情之间的关键区别，然后使用这些区别来指导我们对基于视频的微表情分析进行级联结构的研究调查，包括神经心理学基础、数据集、特征、定位算法、识别算法、应用和最新方法的评估。对于每个方面，我们都讨论和探讨了基本技术、高级发展和主要挑战。此外，在考虑了现有微表情数据集的局限性之后，我们提出并发布了一个新的数据集——称为微表情和宏表情仓库（MMEW），其中包含了更多的视频样本和更多标记的情感类型。然后，我们分别在 CAS(ME)上对有代表性的方法进行了统一的比较，用于定位，以及在 MMEW 和 SAMM 上进行了统一的比较，用于识别。最后，探讨并概述了一些潜在的未来研究方向。
