Al-Saidi Muslem, Ballagi Áron, Hassen Oday Ali, Saad Saad M
Doctoral School of Multidisciplinary Engineering Sciences, Széchenyi István University, Egyetem tér 1, 9026 Gyor, Hungary.
Department of Automation, Széchenyi István University, Egyetem tér 1, 9026 Gyor, Hungary.
Sensors (Basel). 2024 Dec 7;24(23):7828. doi: 10.3390/s24237828.
Uncertainty-aware soft sensors in sign language recognition (SLR) integrate methods to quantify and manage the uncertainty in their predictions. This is particularly crucial in SLR due to the variability in sign language gestures and differences in individual signing styles. Managing uncertainty allows the system to handle variations in signing styles, lighting conditions, and occlusions more effectively. While current techniques for handling uncertainty in SLR systems offer significant benefits in terms of improved accuracy and robustness, they also come with notable disadvantages. High computational complexity, data dependency, scalability issues, sensor and environmental limitations, and real-time constraints all pose significant hurdles. The aim of the work is to develop and evaluate a Type-2 Neutrosophic Hidden Markov Model (HMM) for SLR that leverages the advanced uncertainty handling capabilities of Type-2 neutrosophic sets. In the suggested soft sensor model, the Foot of Uncertainty (FOU) allows Type-2 Neutrosophic HMMs to represent uncertainty as intervals, capturing the range of possible values for truth, falsity, and indeterminacy. This is especially useful in SLR, where gestures can be ambiguous or imprecise. This enhances the model's ability to manage complex uncertainties in sign language gestures and mitigate issues related to model drift. The FOU provides a measure of confidence for each recognition result by indicating the range of uncertainty. By effectively addressing uncertainty and enhancing subject independence, the model can be integrated into real-life applications, improving interactions, learning, and accessibility for the hearing-impaired. Examples such as assistive devices, educational tools, and customer service automation highlight its transformative potential. The experimental evaluation demonstrates the superiority of the Type-2 Neutrosophic HMM over the Type-1 Neutrosophic HMM in terms of accuracy for SLR. Specifically, the Type-2 Neutrosophic HMM consistently outperforms its Type-1 counterpart across various test scenarios, achieving an average accuracy improvement of 10%.
手语识别（SLR）中具有不确定性感知的软传感器集成了量化和管理预测不确定性的方法。由于手语手势的可变性和个体手语风格的差异，这在SLR中尤为关键。管理不确定性使系统能够更有效地处理手语风格、光照条件和遮挡方面的变化。虽然当前SLR系统中处理不确定性的技术在提高准确性和鲁棒性方面有显著优势，但也存在明显缺点。高计算复杂度、数据依赖性、可扩展性问题、传感器和环境限制以及实时约束都构成了重大障碍。这项工作的目的是开发和评估一种用于SLR的二型中性模糊隐马尔可夫模型（HMM），该模型利用二型中性模糊集先进的不确定性处理能力。在所提出的软传感器模型中，不确定性足迹（FOU）允许二型中性模糊HMM将不确定性表示为区间，捕捉真、假和不确定性的可能值范围。这在SLR中特别有用，因为手势可能模糊或不精确。这增强了模型管理手语手势中复杂不确定性的能力，并减轻了与模型漂移相关的问题。FOU通过指示不确定性范围为每个识别结果提供了置信度度量。通过有效解决不确定性并增强主体独立性，该模型可以集成到实际应用中，改善听力障碍者的交互、学习和可达性。辅助设备、教育工具和客户服务自动化等示例突出了其变革潜力。实验评估表明，在SLR的准确性方面，二型中性模糊HMM优于一型中性模糊HMM。具体而言，在各种测试场景中，二型中性模糊HMM始终优于其一型对应模型，平均准确率提高了10%。
