Teachasrisaksakul Krittameth, Wu Liqun, Yang Guang-Zhong, Lo Benny
Annu Int Conf IEEE Eng Med Biol Soc. 2018 Jul;2018:3517-3520. doi: 10.1109/EMBC.2018.8513098.
Dyscalculia is a learning difficulty hindering fundamental arithmetical competence. Children with dyscalculia often have difficulties in engaging in lessons taught with traditional teaching methods. In contrast, an educational game is an attractive alternative. Recent educational studies have shown that gestures could have a positive impact in learning. With the recent development of low cost wearable sensors, a gesture based educational game could be used as a tool to improve the learning outcomes particularly for children with dyscalculia. In this paper, two generic gesture recognition methods are proposed for developing an interactive educational game with wearable inertial sensors. The first method is a multilayered perceptron classifier based on the accelerometer and gyroscope readings to recognize hand gestures. As gyroscope is more power demanding and not all low-cost wearable device has a gyroscope, we have simplified the method using a nearest centroid classifier for classifying hand gestures with only the accelerometer readings. The method has been integrated into open-source educational games. Experimental results based on 5 subjects have demonstrated the accuracy of inertial sensor based hand gesture recognitions. The results have shown that both methods can recognize 15 different hand gestures with the accuracy over 93%.
计算障碍是一种妨碍基本算术能力的学习困难。患有计算障碍的儿童在参与用传统教学方法授课的课程时往往存在困难。相比之下，教育游戏是一种有吸引力的替代方式。最近的教育研究表明，手势可能对学习产生积极影响。随着低成本可穿戴传感器的最新发展，基于手势的教育游戏可以用作提高学习成果的工具，特别是对于患有计算障碍的儿童。在本文中，提出了两种通用的手势识别方法，用于开发一款使用可穿戴惯性传感器的交互式教育游戏。第一种方法是基于加速度计和陀螺仪读数的多层感知器分类器，用于识别手势。由于陀螺仪对电量要求更高，而且并非所有低成本可穿戴设备都配备陀螺仪，我们使用最近邻质心分类器简化了该方法，仅通过加速度计读数对手势进行分类。该方法已集成到开源教育游戏中。基于5名受试者的实验结果证明了基于惯性传感器的手势识别的准确性。结果表明，两种方法都能识别15种不同的手势，准确率超过93%。
