Li Qiming, Huang Chen, Yao Zhengwei, Chen Yimin, Ma Lizhuang
Department of Computer Science and Engineering, Shanghai Jiaotong University, Shanghai, China.
Department of Computer Science and Technology, Shanghai Maritime University, Shanghai, China.
Int J Med Robot. 2018 Oct;14(5):e1931. doi: 10.1002/rcs.1931. Epub 2018 Jun 28.
Human-computer interaction (HCI) is an important feature of augmented reality (AR) technology. The naturalness is the inevitable trend of HCI. Gesture is the most natural and frequently used body auxiliary interaction mode in daily interactions except for language. However, there are often meaningless, subconscious gesture intervals between the two adjacent dynamic gestures. So, continuous dynamic gesture spotting is the premise and basis of dynamic gesture recognition, but there is no mature and unified algorithm to solve this problem.
In order to realize the natural HCI based on gesture recognition entirely, a general AR application development platform is presented in this paper.
According to the position and pose tracking data of the user's hand, the dynamic gesture spotting algorithm based on evidence theory is proposed. Firstly, Through analysis of the speed change of hand motion during the dynamic gestures, three knowledge rules are summed up. Then, accurate dynamic gesture spotting is realized with the application of evidence reasoning. Moreover, this algorithm first detects the starting point of gesture in the rising trend of hand motion speed, eliminates the delay between spotting and recognition, and thus ensures real-time performance. Finally, the algorithm is verified in several AR applications developed on the platform.
There are two main experimental results. First, there are six users participating in the dynamic gesture spotting experiment, and the gesture spotting accuracy can meet the demand. Second, The accuracy of recognition after spotting is higher than that of the simultaneous recognition and spotting.
So, It can be concluded that the proposed continuous dynamic gesture spotting algorithm based on Dempster-Shafer theory can extract almost all the effective dynamic gestures in the HCI of our AR platform, and on this basis, it can effectively improve the accuracy of the subsequent dynamic gesture recognition.
人机交互（HCI）是增强现实（AR）技术的一个重要特征。自然性是人机交互的必然趋势。手势是日常交互中除语言之外最自然且最常用的身体辅助交互方式。然而，在两个相邻的动态手势之间常常存在无意义的、下意识的手势间隔。因此，连续动态手势检测是动态手势识别的前提和基础，但目前尚无成熟统一的算法来解决这一问题。
为了完全实现基于手势识别的自然人机交互，本文提出了一个通用的AR应用开发平台。
根据用户手部的位置和姿态跟踪数据，提出了基于证据理论的动态手势检测算法。首先，通过分析动态手势过程中手部运动的速度变化，总结出三条知识规则。然后，应用证据推理实现精确的动态手势检测。此外，该算法首先在手部运动速度上升趋势中检测手势的起点，消除了检测与识别之间的延迟，从而保证了实时性。最后，在该平台开发的多个AR应用中对算法进行了验证。
主要有两个实验结果。第一，有六名用户参与了动态手势检测实验，手势检测准确率能够满足需求。第二，检测后识别的准确率高于同时进行识别和检测的准确率。
因此，可以得出结论，所提出的基于Dempster-Shafer理论的连续动态手势检测算法能够在我们的AR平台人机交互中提取几乎所有有效的动态手势，并在此基础上有效提高后续动态手势识别的准确率。
