Annu Int Conf IEEE Eng Med Biol Soc. 2022 Jul;2022:4316-4319. doi: 10.1109/EMBC48229.2022.9871491.
Sign language is a means of communication between the deaf community and normal hearing people who use hand gestures, facial expressions, and body language to communicate. It has the same level of complexity as spoken language, but it does not employ the same sentence structure as English. The motions in sign language comprise a range of distinct hand and finger articulations that are occasionally synchronized with the head, face, and body. Existing sign language recognition systems are mainly camera-based, which have fundamental limitations of poor lighting conditions, potential training challenges with longer video sequence data, and serious privacy concerns. This study presents a first of its kind, contact-less and privacy-preserving British sign language (BSL) Recognition system using Radar and deep learning algorithms. Six most common emotions are considered in this proof of concept study, namely confused, depressed, happy, hate, lonely, and sad. The collected data is represented in the form of spectrograms. Three state-of-the-art deep learning models, namely, InceptionV3, VGG19, and VGG16 models then extract spatiotemporal features from the spectrogram. Finally, BSL emotions are accurately identified by classifying the spectrograms into considered emotion signs. Comparative simulation results demonstrate that a maximum classifying accuracy of 93.33% is obtained on all classes using the VGG16 model.
手语是聋人社区与正常听力人群之间进行交流的一种方式，它使用手势、面部表情和身体语言来进行沟通。它与口语具有相同的复杂程度，但它的句子结构与英语不同。手语中的动作包括一系列独特的手部和手指姿势，这些姿势偶尔与头部、面部和身体同步。现有的手语识别系统主要基于摄像机，这些系统存在光照条件差、长视频序列数据训练潜在挑战以及严重的隐私问题等基本局限性。本研究提出了一种首创的、非接触式和保护隐私的基于雷达和深度学习算法的英国手语（BSL）识别系统。本概念验证研究考虑了六种最常见的情绪，即困惑、沮丧、快乐、仇恨、孤独和悲伤。所收集的数据以频谱图的形式表示。然后，三个最先进的深度学习模型（即 InceptionV3、VGG19 和 VGG16 模型）从频谱图中提取时空特征。最后，通过将频谱图分类为考虑的情绪符号，准确识别 BSL 情绪。比较模拟结果表明，使用 VGG16 模型可以在所有类别上获得最高 93.33%的分类准确率。
