GVLAB - University of Agriculture and Technology of Tokyo, Tokyo, Japan.
PLoS One. 2020 Feb 19;15(2):e0228869. doi: 10.1371/journal.pone.0228869. eCollection 2020.
Human activity recognition is an important and difficult topic to study because of the important variability between tasks repeated several times by a subject and between subjects. This work is motivated by providing time-series signal classification and a robust validation and test approaches. This study proposes to classify 60 signs from the American Sign Language based on data provided by the LeapMotion sensor by using different conventional machine learning and deep learning models including a model called DeepConvLSTM that integrates convolutional and recurrent layers with Long-Short Term Memory cells. A kinematic model of the right and left forearm/hand/fingers/thumb is proposed as well as the use of a simple data augmentation technique to improve the generalization of neural networks. DeepConvLSTM and convolutional neural network demonstrated the highest accuracy compared to other models with 91.1 (3.8) and 89.3 (4.0) % respectively compared to the recurrent neural network or multi-layer perceptron. Integrating convolutional layers in a deep learning model seems to be an appropriate solution for sign language recognition with depth sensors data.
人类活动识别是一个重要且困难的研究课题，因为受试者多次重复的任务之间以及受试者之间存在重要的可变性。这项工作的动机是提供时间序列信号分类以及稳健的验证和测试方法。本研究提出了一种基于 LeapMotion 传感器提供的数据对美国手语中的 60 个手势进行分类的方法，使用了不同的传统机器学习和深度学习模型，包括一种称为 DeepConvLSTM 的模型，该模型将卷积和循环层与长短时记忆单元相结合。还提出了一种右前臂/手/手指/拇指的运动学模型，并使用简单的数据增强技术来提高神经网络的泛化能力。与其他模型相比，DeepConvLSTM 和卷积神经网络表现出了最高的准确率，分别为 91.1（3.8）%和 89.3（4.0）%，而递归神经网络或多层感知机的准确率为 86.5（4.4）%。在深度学习模型中集成卷积层似乎是一种利用深度传感器数据进行手语识别的合适解决方案。
