Alasmari Naif, Asiri Sultan
Computer Science Department, Applied College, Muhayil, King Khalid University, Abha, Saudi Arabia.
Center for Artificial Intelligence (CAI), King Khalid University, Abha, 61421, Saudi Arabia.
Sci Rep. 2025 May 23;15(1):18012. doi: 10.1038/s41598-025-01588-w.
Sign languages are essential for communication among over 430 million deaf and hard-of-hearing individuals worldwide. However, recognizing Arabic Sign Language (ArSL) in real-world settings remains challenging due to issues like background noise, lighting variations, and hand occlusions. These limitations hinder the effectiveness of existing systems in applications such as assistive technologies and education. To tackle these challenges, we propose ASLDetect, a new model for ArSL recognition that leverages ResNet for feature extraction and a U-Net-based architecture for accurate gesture segmentation. Our method includes preprocessing steps like resizing images to 64 64 pixels, normalization, and selective augmentation to improve robustness in diverse environments. We evaluated ASLDetect on two datasets: ArASL2018, which features plain backgrounds, and ArASL2021, which includes more complex and diverse environments. On ArASL2018, ASLDetect achieved an accuracy of 99.35%, surpassing ResNet34 (99.08%), T-SignSys (97.92%), and UrSL-CNN (0.98%). For ArASL2021, we applied transfer learning from our ArASL2018-trained model, significantly improving performance and reaching 86.84% accuracy-outperforming ResNet34 (82.5%), T-SignSys (58.98%), and UrSL-CNN (0.49%). These results highlight ASLDetect's accuracy, robustness, and adaptability.
手语对于全球超过4.3亿失聪和听力障碍人士之间的交流至关重要。然而，在现实场景中识别阿拉伯手语（ArSL）仍然具有挑战性，因为存在背景噪音、光照变化和手部遮挡等问题。这些限制阻碍了现有系统在辅助技术和教育等应用中的有效性。为了应对这些挑战，我们提出了ASLDetect，一种用于ArSL识别的新模型，它利用ResNet进行特征提取，并采用基于U-Net的架构进行精确的手势分割。我们的方法包括预处理步骤，如图像调整为64×64像素、归一化和选择性增强，以提高在不同环境中的鲁棒性。我们在两个数据集上评估了ASLDetect：具有纯色背景的ArASL2018和包含更复杂多样环境的ArASL2021。在ArASL2018上，ASLDetect的准确率达到99.35%，超过了ResNet34（99.08%）、T-SignSys（97.92%）和UrSL-CNN（0.98%）。对于ArASL2021，我们应用了从在ArASL2018上训练的模型进行迁移学习，显著提高了性能，准确率达到86.84%，超过了ResNet34（82.5%）、T-SignSys（58.98%）和UrSL-CNN（0.49%）。这些结果突出了ASLDetect的准确性、鲁棒性和适应性。
