Institute of Medical Systems Biology, Ulm University, Albert-Einstein-Allee 11, 89081 Ulm, Germany.
Institute of Neural Information Processing, Ulm University, James-Frank-Ring, 89081 Ulm, Germany.
Sensors (Basel). 2020 Feb 4;20(3):839. doi: 10.3390/s20030839.
Several approaches have been proposed for the analysis of pain-related facial expressions. These approaches range from common classification architectures based on a set of carefully designed handcrafted features, to deep neural networks characterised by an autonomous extraction of relevant facial descriptors and simultaneous optimisation of a classification architecture. In the current work, an end-to-end approach based on attention networks for the analysis and recognition of pain-related facial expressions is proposed. The method combines both spatial and temporal aspects of facial expressions through a weighted aggregation of attention-based neural networks' outputs, based on sequences of Motion History Images (MHIs) and Optical Flow Images (OFIs). Each input stream is fed into a specific attention network consisting of a Convolutional Neural Network (CNN) coupled to a Bidirectional Long Short-Term Memory (BiLSTM) Recurrent Neural Network (RNN). An attention mechanism generates a single weighted representation of each input stream (MHI sequence and OFI sequence), which is subsequently used to perform specific classification tasks. Simultaneously, a weighted aggregation of the classification scores specific to each input stream is performed to generate a final classification output. The assessment conducted on both the and points at the relevance of the proposed approach, as its classification performance is on par with state-of-the-art classification approaches proposed in the literature.
已经提出了几种用于分析与疼痛相关的面部表情的方法。这些方法范围从基于一组精心设计的手工制作特征的常见分类架构，到具有自主提取相关面部描述符和同时优化分类架构的深度神经网络。在当前的工作中，提出了一种基于注意力网络的端到端方法，用于分析和识别与疼痛相关的面部表情。该方法通过基于运动历史图像（MHI）和光流图像（OFIs）序列的基于注意力的神经网络输出的加权聚合，结合了面部表情的空间和时间方面。每个输入流都被馈送到特定的注意力网络中，该网络由卷积神经网络（CNN）与双向长短期记忆（BiLSTM）递归神经网络（RNN）耦合而成。注意力机制为每个输入流（MHI 序列和 OFI 序列）生成单个加权表示，随后用于执行特定的分类任务。同时，对每个输入流的分类分数进行加权聚合，以生成最终的分类输出。在 和 点上进行的评估表明了所提出方法的相关性，因为其分类性能与文献中提出的最先进的分类方法相当。
