Katthi Jaswanth Reddy, Ganapathy Sriram, Kothinti Sandeep, Slaney Malcolm
Annu Int Conf IEEE Eng Med Biol Soc. 2020 Jul;2020:3505-3508. doi: 10.1109/EMBC44109.2020.9176208.
The process of decoding the auditory brain for an acoustic stimulus involves finding the relationship between the audio input and the brain activity measured in terms of Electroencephalography (EEG) recordings. Prior methods focus on linear analysis methods like Canonical Correlation Analysis (CCA) to establish a relationship. In this paper, we present a deep learning framework that is learned to maximize correlation. For dealing with high levels of noise in EEG data, we employ regularization techniques and experiment with various model architectures. With a paired dataset of audio envelope and EEG, we perform several experiments with deep correlation analysis using forward and backward correlation models. In these experiments, we show that regularized deep CCA is consistently able to outperform the linear models in terms of providing improved correlation (up to 9% absolute improvement in Pearson correlation which is statistically significant). We present an analysis that highlights the benefits of using dropouts for neural network regularization in the deep CCA model.Clinical relevance - The proposed method helps to decode human auditory attention. In the case of overlapping speech from two speakers, decoding the auditory attention provides information about how well the sources are separated in the brain and which of the sources is attended. This can impact cochlear implants that use EEG for decoding attention as well as in development of BCI applications. The correlation method proposed in this work can also be extended to other modalities like visual stimuli.
对听觉刺激进行听觉脑解码的过程涉及到寻找音频输入与通过脑电图（EEG）记录测量的大脑活动之间的关系。先前的方法侧重于像典型相关分析（CCA）这样的线性分析方法来建立关系。在本文中，我们提出了一个深度学习框架，该框架通过学习来最大化相关性。为了处理EEG数据中的高噪声水平，我们采用正则化技术并对各种模型架构进行实验。利用音频包络和EEG的配对数据集，我们使用前向和后向相关模型进行了几次深度相关分析实验。在这些实验中，我们表明正则化深度CCA在提供改进的相关性方面始终能够优于线性模型（皮尔逊相关性绝对提高高达9%，具有统计学意义）。我们进行了一项分析，突出了在深度CCA模型中使用随机失活进行神经网络正则化的好处。临床相关性——所提出的方法有助于解码人类的听觉注意力。在两个说话者重叠语音的情况下，解码听觉注意力可提供有关大脑中声源分离程度以及所关注声源的信息。这会影响使用EEG来解码注意力的人工耳蜗以及脑机接口应用的开发。本文提出的相关方法还可以扩展到视觉刺激等其他模态。
