Chengaiyan Sandhya, Retnapandian Anandha Sree, Anandan Kavitha
Department of Biomedical Engineering, Centre for Healthcare Technologies, SSN College of Engineering, Chennai, Tamilnadu India.
Cogn Neurodyn. 2020 Feb;14(1):1-19. doi: 10.1007/s11571-019-09558-5. Epub 2019 Oct 4.
Retrieval of unintelligible speech is a basic need for speech impaired and is under research for several decades. But retrieval of random words from thoughts needs a substantial and consistent approach. This work focuses on the preliminary steps of retrieving vowels from Electroencephalography (EEG) signals acquired while speaking and imagining of speaking a consonant-vowel-consonant (CVC) word. The process, referred to as Speech imagery is imagining of speaking to oneself silently in the mind. Speech imagery is a form of mental imagery. Brain connectivity estimators such as EEG coherence, Partial Directed Coherence, Directed Transfer Function and Transfer Entropy have been used to estimate the concurrency and causal dependence (direction and strength) between different brain regions. From brain connectivity results it has been observed that the left frontal and left temporal electrodes were activated for speech and speech imagery processes. These brain connectivity estimators have been used for training Recurrent Neural Networks (RNN) and Deep Belief Networks (DBN) for identifying the vowel from the subject's thought. Though the accuracy level was found to be varying for each vowel while speaking and imagining of speaking the CVC word, the overall classification accuracy was found to be 72% while using RNN whereas a classification accuracy of 80% was observed while using DBN. DBN was found to outperform RNN in both the speech and speech imagery processes. Thus, the combination of brain connectivity estimators and deep learning techniques appear to be effective in identifying the vowel from EEG signals of subjects' thought.
恢复难以理解的语音是言语障碍者的基本需求，并且已经研究了几十年。但是从思维中检索随机单词需要一种实质性且一致的方法。这项工作专注于从在说出和想象说出一个辅音-元音-辅音（CVC）单词时采集的脑电图（EEG）信号中检索元音的初步步骤。这个过程，即言语意象，是指在脑海中默默地对自己说话。言语意象是心理意象的一种形式。诸如EEG相干性、偏定向相干性、定向传递函数和转移熵等脑连接性估计器已被用于估计不同脑区之间的并发情况和因果依赖性（方向和强度）。从脑连接性结果可以观察到，左额叶和左颞叶电极在言语和言语意象过程中被激活。这些脑连接性估计器已被用于训练递归神经网络（RNN）和深度信念网络（DBN），以从受试者的思维中识别元音。虽然在说出和想象说出CVC单词时，每个元音的准确率各不相同，但使用RNN时总体分类准确率为72%，而使用DBN时观察到的分类准确率为80%。发现在言语和言语意象过程中DBN都优于RNN。因此，脑连接性估计器和深度学习技术的结合似乎在从受试者思维的EEG信号中识别元音方面是有效的。
