Artificial Intelligence and Computer Vision Research Lab, Escuela Politécnica Nacional, Quito 170517, Ecuador.
Faculty of Engineering, Universidad Andres Bello, Santiago, Chile.
Sensors (Basel). 2023 Apr 12;23(8):3905. doi: 10.3390/s23083905.
In recent years, hand gesture recognition (HGR) technologies that use electromyography (EMG) signals have been of considerable interest in developing human-machine interfaces. Most state-of-the-art HGR approaches are based mainly on supervised machine learning (ML). However, the use of reinforcement learning (RL) techniques to classify EMGs is still a new and open research topic. Methods based on RL have some advantages such as promising classification performance and online learning from the user's experience. In this work, we propose a user-specific HGR system based on an RL-based agent that learns to characterize EMG signals from five different hand gestures using Deep Q-network (DQN) and Double-Deep Q-Network (Double-DQN) algorithms. Both methods use a feed-forward artificial neural network (ANN) for the representation of the agent policy. We also performed additional tests by adding a long-short-term memory (LSTM) layer to the ANN to analyze and compare its performance. We performed experiments using training, validation, and test sets from our public dataset, EMG-EPN-612. The final accuracy results demonstrate that the best model was DQN without LSTM, obtaining classification and recognition accuracies of up to 90.37%±10.7% and 82.52%±10.9%, respectively. The results obtained in this work demonstrate that RL methods such as DQN and Double-DQN can obtain promising results for classification and recognition problems based on EMG signals.
近年来，使用肌电图 (EMG) 信号的手势识别 (HGR) 技术在开发人机界面方面引起了相当大的兴趣。大多数最先进的 HGR 方法主要基于监督机器学习 (ML)。然而，使用强化学习 (RL) 技术对 EMG 进行分类仍然是一个新的开放研究课题。基于 RL 的方法具有一些优势，例如有前途的分类性能和从用户经验中进行在线学习。在这项工作中，我们提出了一个基于 RL 代理的特定于用户的 HGR 系统，该代理使用深度 Q 网络 (DQN) 和双深度 Q 网络 (Double-DQN) 算法学习从五个不同的手势中对 EMG 信号进行特征化。这两种方法都使用前馈人工神经网络 (ANN) 来表示代理策略。我们还通过在 ANN 中添加长短期记忆 (LSTM) 层来进行额外的测试，以分析和比较其性能。我们使用来自我们的公共数据集 EMG-EPN-612 的训练、验证和测试集进行实验。最终的准确性结果表明，最好的模型是没有 LSTM 的 DQN，分别获得高达 90.37%±10.7%和 82.52%±10.9%的分类和识别准确率。这项工作的结果表明，DQN 和 Double-DQN 等 RL 方法可以为基于 EMG 信号的分类和识别问题获得有前途的结果。
