Department of Electronic Engineering, Tsinghua University, Beijing 100084, China.
AI Lab, Bytedance Inc., Beijing 100086, China.
Sensors (Basel). 2019 Jan 10;19(2):239. doi: 10.3390/s19020239.
Dynamic hand gesture recognition has attracted increasing attention because of its importance for human⁻computer interaction. In this paper, we propose a novel motion feature augmented network (MFA-Net) for dynamic hand gesture recognition from skeletal data. MFA-Net exploits motion features of finger and global movements to augment features of deep network for gesture recognition. To describe finger articulated movements, finger motion features are extracted from the hand skeleton sequence via a variational autoencoder. Global motion features are utilized to represent the global movements of hand skeleton. These motion features along with the skeleton sequence are then fed into three branches of a recurrent neural network (RNN), which augment the motion features for RNN and improve the classification performance. The proposed MFA-Net is evaluated on two challenging skeleton-based dynamic hand gesture datasets, including DHG-14/28 dataset and SHREC'17 dataset. Experimental results demonstrate that our proposed method achieves comparable performance on DHG-14/28 dataset and better performance on SHREC'17 dataset when compared with start-of-the-art methods.
动态手势识别由于在人机交互中的重要性而受到越来越多的关注。在本文中，我们提出了一种新颖的运动特征增强网络（MFA-Net），用于从骨骼数据中进行动态手势识别。MFA-Net 利用手指和全局运动的运动特征来增强手势识别的深度网络特征。为了描述手指的关节运动，从手部骨骼序列中通过变分自编码器提取手指运动特征。全局运动特征用于表示手部骨骼的全局运动。这些运动特征以及骨骼序列随后被输入到递归神经网络（RNN）的三个分支中，从而增强了 RNN 的运动特征并提高了分类性能。所提出的 MFA-Net 在两个具有挑战性的基于骨骼的动态手势数据集上进行了评估，包括 DHG-14/28 数据集和 SHREC'17 数据集。实验结果表明，与最先进的方法相比，我们提出的方法在 DHG-14/28 数据集上具有可比的性能，在 SHREC'17 数据集上具有更好的性能。
