Research Laboratory LaTICE, University of Tunis, Tunis 1008, Tunisia.
Mada-Assistive Technology Center Qatar, Doha P.O. Box 24230, Qatar.
Sensors (Basel). 2023 Oct 9;23(19):8343. doi: 10.3390/s23198343.
The analysis and recognition of sign languages are currently active fields of research focused on sign recognition. Various approaches differ in terms of analysis methods and the devices used for sign acquisition. Traditional methods rely on video analysis or spatial positioning data calculated using motion capture tools. In contrast to these conventional recognition and classification approaches, electromyogram (EMG) signals, which measure muscle electrical activity, offer potential technology for detecting gestures. These EMG-based approaches have recently gained attention due to their advantages. This prompted us to conduct a comprehensive study on the methods, approaches, and projects utilizing EMG sensors for sign language handshape recognition. In this paper, we provided an overview of the sign language recognition field through a literature review, with the objective of offering an in-depth review of the most significant techniques. These techniques were categorized in this article based on their respective methodologies. The survey discussed the progress and challenges in sign language recognition systems based on surface electromyography (sEMG) signals. These systems have shown promise but face issues like sEMG data variability and sensor placement. Multiple sensors enhance reliability and accuracy. Machine learning, including deep learning, is used to address these challenges. Common classifiers in sEMG-based sign language recognition include SVM, ANN, CNN, KNN, HMM, and LSTM. While SVM and ANN are widely used, random forest and KNN have shown better performance in some cases. A multilayer perceptron neural network achieved perfect accuracy in one study. CNN, often paired with LSTM, ranks as the third most popular classifier and can achieve exceptional accuracy, reaching up to 99.6% when utilizing both EMG and IMU data. LSTM is highly regarded for handling sequential dependencies in EMG signals, making it a critical component of sign language recognition systems. In summary, the survey highlights the prevalence of SVM and ANN classifiers but also suggests the effectiveness of alternative classifiers like random forests and KNNs. LSTM emerges as the most suitable algorithm for capturing sequential dependencies and improving gesture recognition in EMG-based sign language recognition systems.
手语的分析和识别是目前专注于手语识别的研究热点领域。各种方法在分析方法和用于获取手语的设备方面有所不同。传统方法依赖于视频分析或使用运动捕捉工具计算的空间定位数据。与这些传统的识别和分类方法相比，肌电图 (EMG) 信号可以测量肌肉的电活动，为检测手势提供了潜在的技术。由于具有这些优势，基于 EMG 的方法最近引起了关注。这促使我们对手语手形识别中使用 EMG 传感器的方法、方法和项目进行了全面研究。在本文中，我们通过文献综述对手语识别领域进行了概述，旨在深入回顾最重要的技术。本文根据各自的方法对这些技术进行了分类。该调查讨论了基于表面肌电图 (sEMG) 信号的手语识别系统的进展和挑战。这些系统具有很大的前景，但面临着 sEMG 数据变异性和传感器放置等问题。多个传感器提高了可靠性和准确性。机器学习，包括深度学习，用于解决这些挑战。sEMG 手语识别中的常见分类器包括 SVM、ANN、CNN、KNN、HMM 和 LSTM。虽然 SVM 和 ANN 被广泛使用，但在某些情况下，随机森林和 KNN 的性能更好。在一项研究中，多层感知器神经网络实现了完美的准确率。CNN 通常与 LSTM 结合使用，是第三大最受欢迎的分类器，可以达到极高的准确率，当同时使用 EMG 和 IMU 数据时，准确率可高达 99.6%。LSTM 在手语识别系统中因其能够处理 EMG 信号中的顺序依赖性而备受推崇，是其关键组成部分。总的来说，该调查突出了 SVM 和 ANN 分类器的普遍性，但也表明了替代分类器（如随机森林和 KNN）的有效性。LSTM 是一种最适合捕获顺序依赖性并提高基于 EMG 的手语识别系统中手势识别的算法。
