Defense Innovation Institute, Academy of Military Sciences (AMS), Beijing 100071, People's Republic of China.
Intelligent Game and Decision Laboratory, Beijing 100071, People's Republic of China.
J Neural Eng. 2024 Sep 3;21(5). doi: 10.1088/1741-2552/ad7321.
. The decline in the performance of electromyography (EMG)-based silent speech recognition is widely attributed to disparities in speech patterns, articulation habits, and individual physiology among speakers. Feature alignment by learning a discriminative network that resolves domain offsets across speakers is an effective method to address this problem. The prevailing adversarial network with a branching discriminator specializing in domain discrimination renders insufficiently direct contribution to categorical predictions of the classifier.. To this end, we propose a simplified discrepancy-based adversarial network with a streamlined end-to-end structure for EMG-based cross-subject silent speech recognition. Highly aligned features across subjects are obtained by introducing a Nuclear-norm Wasserstein discrepancy metric on the back end of the classification network, which could be utilized for both classification and domain discrimination. Given the low-level and implicitly noisy nature of myoelectric signals, we devise a cascaded adaptive rectification network as the front-end feature extraction network, adaptively reshaping the intermediate feature map with automatically learnable channel-wise thresholds. The resulting features effectively filter out domain-specific information between subjects while retaining domain-invariant features critical for cross-subject recognition.. A series of sentence-level classification experiments with 100 Chinese sentences demonstrate the efficacy of our method, achieving an average accuracy of 89.46% tested on 40 new subjects by training with data from 60 subjects. Especially, our method achieves a remarkable 10.07% improvement compared to the state-of-the-art model when tested on 10 new subjects with 20 subjects employed for training, surpassing its result even with three times training subjects.. Our study demonstrates an improved classification performance of the proposed adversarial architecture using cross-subject myoelectric signals, providing a promising prospect for EMG-based speech interactive application.
基于肌电图（EMG）的无声语音识别性能的下降，广泛归因于说话者之间的语音模式、发音习惯和个体生理差异。通过学习一个能够解决说话者之间域偏移的判别网络来对齐特征是解决该问题的有效方法。现有的带有专门用于域判别分支判别器的对抗网络，对分类器的类别预测贡献不足。为此，我们提出了一种简化的基于差异的对抗网络，具有简化的端到端结构，用于基于 EMG 的跨主体无声语音识别。通过在分类网络的后端引入核范数 Wasserstein 差异度量，可以在分类和域判别两个方面获得高度对齐的特征。鉴于肌电信号的低水平和隐含噪声性质，我们设计了一个级联自适应整流网络作为前端特征提取网络，自适应地使用自动学习的通道级阈值重新塑造中间特征图。所得到的特征有效地滤除了主体之间的特定于域的信息，同时保留了对跨主体识别至关重要的域不变特征。一系列使用 100 个中文句子的句子级分类实验表明，我们的方法具有有效性，在 60 个主体的训练数据上测试 40 个新主体时，平均准确率为 89.46%。特别是，当在 10 个新主体上测试，而训练主体为 20 个时，我们的方法比最先进的模型提高了 10.07%，甚至超过了其在 3 倍训练主体上的结果。我们的研究表明，所提出的对抗架构使用跨主体肌电信号可以提高分类性能，为基于 EMG 的语音交互应用提供了有前途的前景。
