Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China.
University of Science and Technology of China, Hefei 230026, China.
Sensors (Basel). 2021 Jan 27;21(3):833. doi: 10.3390/s21030833.
Recognizing facial expression has attracted much more attention due to its broad range of applications in human-computer interaction systems. Although facial representation is crucial to final recognition accuracy, traditional handcrafted representations only reflect shallow characteristics and it is uncertain whether the convolutional layer can extract better ones. In addition, the policy that weights are shared across a whole image is improper for structured face images. To overcome such limitations, a novel method based on patches of interest, the Patch Attention Layer (PAL) of embedding handcrafted features, is proposed to learn the local shallow facial features of each patch on face images. Firstly, a handcrafted feature, Gabor surface feature (GSF), is extracted by convolving the input face image with a set of predefined Gabor filters. Secondly, the generated feature is segmented as nonoverlapped patches that can capture local shallow features by the strategy of using different local patches with different filters. Then, the weighted shallow features are fed into the remaining convolutional layers to capture high-level features. Our method can be carried out directly on a static image without facial landmark information, and the preprocessing step is very simple. Experiments on four databases show that our method achieved very competitive performance (Extended Cohn-Kanade database (CK+): 98.93%; Oulu-CASIA: 97.57%; Japanese Female Facial Expressions database (JAFFE): 93.38%; and RAF-DB: 86.8%) compared to other state-of-the-art methods.
由于在人机交互系统中的广泛应用，面部表情识别受到了越来越多的关注。尽管面部表示对于最终识别精度至关重要，但传统的手工制作表示仅反映了浅层特征，并且不确定卷积层是否可以提取更好的特征。此外，权重在整个图像上共享的策略对于结构化面部图像是不适当的。为了克服这些限制，提出了一种基于感兴趣区域的patch 的新方法，即嵌入手工制作特征的 Patch Attention Layer (PAL)，用于学习面部图像上每个 patch 的局部浅层面部特征。首先，通过用一组预定义的 Gabor 滤波器卷积输入的面部图像，提取手工制作的特征，即 Gabor 表面特征（GSF）。其次，通过使用具有不同滤波器的不同局部 patch 的策略，将生成的特征分割为非重叠 patch，从而可以捕获局部浅层特征。然后，将加权的浅层特征馈送到剩余的卷积层中以捕获高层特征。我们的方法可以直接在没有面部地标信息的静态图像上进行，预处理步骤非常简单。在四个数据库上的实验表明，与其他最先进的方法相比，我们的方法取得了非常有竞争力的性能（扩展的 Cohn-Kanade 数据库（CK+）：98.93%；Oulu-CASIA：97.57%；日本女性面部表情数据库（JAFFE）：93.38%；和 RAF-DB：86.8%）。
