School of Automation, China University of Geosciences, Wuhan 430074, China.
Hubei Key Laboratory of Advanced Control and Intelligent Automation for Complex Systems, Wuhan 430074, China.
Sensors (Basel). 2021 Feb 2;21(3):1007. doi: 10.3390/s21031007.
Hand gesture recognition and hand pose estimation are two closely correlated tasks. In this paper, we propose a deep-learning based approach which jointly learns an intermediate level shared feature for these two tasks, so that the hand gesture recognition task can be benefited from the hand pose estimation task. In the training process, a semi-supervised training scheme is designed to solve the problem of lacking proper annotation. Our approach detects the foreground hand, recognizes the hand gesture, and estimates the corresponding 3D hand pose simultaneously. To evaluate the hand gesture recognition performance of the state-of-the-arts, we propose a challenging hand gesture recognition dataset collected in unconstrained environments. Experimental results show that, the gesture recognition accuracy of ours is significantly boosted by leveraging the knowledge learned from the hand pose estimation task.
手势识别和手姿势估计是两个密切相关的任务。在本文中，我们提出了一种基于深度学习的方法，该方法共同学习这两个任务的中间共享特征，以便手势识别任务可以从手姿势估计任务中受益。在训练过程中，设计了一种半监督训练方案来解决缺乏适当注释的问题。我们的方法可以同时检测前景手、识别手势和估计相应的 3D 手姿势。为了评估最先进的手姿势识别性能，我们提出了一个在非约束环境中收集的具有挑战性的手姿势识别数据集。实验结果表明，通过利用从手姿势估计任务中学到的知识，我们的手势识别精度得到了显著提高。
