van der Maaten Laurens, Hendriks Emile
Pattern Recognition & Bio-informatics Laboratory, Delft University of Technology, Mekelweg 4, 2628 CD Delft, The Netherlands.
Cogn Process. 2012 Oct;13 Suppl 2(Suppl 2):507-18. doi: 10.1007/s10339-011-0419-7. Epub 2011 Oct 12.
In this paper, we investigate to what extent modern computer vision and machine learning techniques can assist social psychology research by automatically recognizing facial expressions. To this end, we develop a system that automatically recognizes the action units defined in the facial action coding system (FACS). The system uses a sophisticated deformable template, which is known as the active appearance model, to model the appearance of faces. The model is used to identify the location of facial feature points, as well as to extract features from the face that are indicative of the action unit states. The detection of the presence of action units is performed by a time series classification model, the linear-chain conditional random field. We evaluate the performance of our system in experiments on a large data set of videos with posed and natural facial expressions. In the experiments, we compare the action units detected by our approach with annotations made by human FACS annotators. Our results show that the agreement between the system and human FACS annotators is higher than 90% and underlines the potential of modern computer vision and machine learning techniques to social psychology research. We conclude with some suggestions on how systems like ours can play an important role in research on social signals.
在本文中，我们研究现代计算机视觉和机器学习技术在通过自动识别面部表情辅助社会心理学研究方面能达到何种程度。为此，我们开发了一个系统，该系统能自动识别面部动作编码系统（FACS）中定义的动作单元。该系统使用一种复杂的可变形模板，即主动外观模型，来对面部外观进行建模。该模型用于识别面部特征点的位置，并从面部提取指示动作单元状态的特征。动作单元存在与否的检测由时间序列分类模型——线性链条件随机场来执行。我们在一个包含摆拍和自然面部表情的视频大数据集上进行实验，评估我们系统的性能。在实验中，我们将我们方法检测到的动作单元与人类FACS注释者所做的注释进行比较。我们的结果表明，该系统与人类FACS注释者之间的一致性高于90%，这凸显了现代计算机视觉和机器学习技术在社会心理学研究中的潜力。我们最后就像我们这样的系统如何能在社会信号研究中发挥重要作用提出了一些建议。
