Alabduallah Bayan, Al Dayil Reham, Alkharashi Abdulwhab, Alneil Amani A
Department of Information Systems, College of Computer and Information Sciences, Princess Nourah Bint Abdulrahman University, P.O. Box 84428, 11671, Riyadh, Saudi Arabia.
Applied College, Imam Mohammad Ibn Saud Islamic University (IMSIU), 11432, Riyadh, Saudi Arabia.
Sci Rep. 2025 Mar 18;15(1):9320. doi: 10.1038/s41598-025-93559-4.
Sign language (SL) is an effective mode of communication, which uses visual-physical methods like hand signals, expressions, and body actions to communicate between the difficulty of hearing and the deaf community, produce opinions, and carry significant conversations. SL recognition (SLR), the procedure of automatically identifying and construing gestures of SL, has gotten considerable attention recently owing to its latent link to the lack of communication between the deaf and the hearing world. Hand gesture detection is its domain, in which computer vision (CV) and artificial intelligence (AI) help deliver non-verbal communication between computers and humans by classifying the significant movements of the human hands. The emergence and constant growth of DL approaches have delivered motivation and momentum for evolving SLR. Therefore, this manuscript presents an Innovative Sign Language Recognition using Hand Pose with Hybrid Metaheuristic Optimization Algorithms in Deep Learning (ISLRHP-HMOADL) technique for Hearing-Impaired Persons. The main objective of the ISLRHP-HMOADL technique focused on hand pose recognition to improve the efficiency and accuracy of sign interpretation for hearing-impaired persons. Initially, the ISLRHP-HMOADL model performs image pre-processing using a wiener filter (WF) to enhance image quality by reducing noise. Furthermore, the fusion of three models, ResNeXt101, VGG19, and vision transformer (ViT), is employed for feature extraction to capture diverse and intricate spatial and contextual details from the images. The bidirectional gated recurrent unit (BiGRU) classifier is implemented for hand pose recognition. To further optimize the performance of the model, the ISLRHP-HMOADL model implements the hybrid crow search-improved grey wolf optimization (CS-IGWO) model for parameter tuning, achieving a finely-tuned configuration that enhances classification accuracy and robustness. A comprehensive experimental study is accomplished under the ASL alphabet dataset to exhibit the improved performance of the ISLRHP-HMOADL model. The comparative results of the ISLRHP-HMOADL model illustrated a superior accuracy value of 99.57% over existing techniques.
手语（SL）是一种有效的交流方式，它使用诸如手势、表情和身体动作等视觉物理方法，在听力障碍者和聋人社区之间进行交流、表达观点并进行重要对话。手语识别（SLR）是自动识别和解释手语手势的过程，由于其与聋人和听力世界之间沟通障碍的潜在联系，最近受到了相当大的关注。手势检测是其领域，其中计算机视觉（CV）和人工智能（AI）通过对人类手部的重要动作进行分类，帮助实现计算机与人类之间的非语言交流。深度学习（DL）方法的出现和不断发展为手语识别的发展提供了动力和契机。因此，本文提出了一种用于听力障碍者的深度学习中基于手部姿态与混合元启发式优化算法的创新手语识别（ISLRHP-HMOADL）技术。ISLRHP-HMOADL技术的主要目标集中在手部姿态识别上，以提高听力障碍者手语解释的效率和准确性。最初，ISLRHP-HMOADL模型使用维纳滤波器（WF）进行图像预处理，通过减少噪声来提高图像质量。此外，采用ResNeXt101、VGG19和视觉Transformer（ViT）这三种模型的融合进行特征提取，以从图像中捕获多样且复杂的空间和上下文细节。双向门控循环单元（BiGRU）分类器用于手部姿态识别。为了进一步优化模型性能，ISLRHP-HMOADL模型采用混合乌鸦搜索-改进灰狼优化（CS-IGWO）模型进行参数调整，实现精细调整的配置，提高分类准确性和鲁棒性。在美式手语字母数据集下完成了全面的实验研究，以展示ISLRHP-HMOADL模型的改进性能。ISLRHP-HMOADL模型的比较结果表明，与现有技术相比，其准确率高达99.57%。
