Department of Electronics Engineering, Keimyung University, Daegu 42601, Korea.
Sensors (Basel). 2018 Oct 19;18(10):3554. doi: 10.3390/s18103554.
Sign language is intentionally designed to allow deaf and dumb communities to convey messages and to connect with society. Unfortunately, learning and practicing sign language is not common among society; hence, this study developed a sign language recognition prototype using the Leap Motion Controller (LMC). Many existing studies have proposed methods for incomplete sign language recognition, whereas this study aimed for full American Sign Language (ASL) recognition, which consists of 26 letters and 10 digits. Most of the ASL letters are static (no movement), but certain ASL letters are dynamic (they require certain movements). Thus, this study also aimed to extract features from finger and hand motions to differentiate between the static and dynamic gestures. The experimental results revealed that the sign language recognition rates for the 26 letters using a support vector machine (SVM) and a deep neural network (DNN) are 80.30% and 93.81%, respectively. Meanwhile, the recognition rates for a combination of 26 letters and 10 digits are slightly lower, approximately 72.79% for the SVM and 88.79% for the DNN. As a result, the sign language recognition system has great potential for reducing the gap between deaf and dumb communities and others. The proposed prototype could also serve as an interpreter for the deaf and dumb in everyday life in service sectors, such as at the bank or post office.
手语是为了让聋哑人群体能够传达信息并与社会联系而有意设计的。不幸的是，在社会中，学习和练习手语并不常见；因此，本研究使用 Leap Motion Controller（LMC）开发了一种手语识别原型。许多现有研究已经提出了用于不完整手语识别的方法，而本研究旨在实现完整的美国手语（ASL）识别，其中包括 26 个字母和 10 个数字。大多数 ASL 字母是静态的（没有运动），但某些 ASL 字母是动态的（它们需要特定的运动）。因此，本研究还旨在从手指和手部运动中提取特征，以区分静态和动态手势。实验结果表明，使用支持向量机（SVM）和深度神经网络（DNN）对 26 个字母进行手语识别的准确率分别为 80.30%和 93.81%。同时，26 个字母和 10 个数字的组合识别率略低，SVM 的识别率约为 72.79%，DNN 的识别率约为 88.79%。因此，手语识别系统在缩小聋哑人群体与其他人之间的差距方面具有很大的潜力。该原型还可以作为聋哑人在银行或邮局等服务行业日常生活中的翻译器。
