Niu Yixiang, Chen Ning, Zhu Hongqing, Jin Jing, Li Guangqiang
School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, China.
School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, China.
Neurosci Lett. 2024 Jan 1;818:137534. doi: 10.1016/j.neulet.2023.137534. Epub 2023 Oct 21.
Music-oriented auditory attention detection (AAD) aims at determining which instrument in polyphonic music a listener is paying attention to by analyzing the listener's electroencephalogram (EEG). However, the existing linear models cannot effectively mimic the nonlinearity of the human brain, resulting in limited performance. Thus, a nonlinear music-oriented AAD model is proposed in this paper. Firstly, an auditory feature and a musical feature are fused to represent musical sources precisely and comprehensively. Secondly, the EEG is enhanced if music stimuli are presented in stereo. Thirdly, a neural network architecture is constructed to capture nonlinear and dynamic interactions between the EEG and auditory stimuli. Finally, the musical source most similar to the EEG in the common embedding space is identified as the attended one. Experimental results demonstrate that the proposed model outperforms all baseline models. On 1-s decision windows, it reaches accuracies of 92.6% and 81.7% under mono duo and trio stimuli, respectively. Additionally, it can be easily extended to speech-oriented AAD. This work can open up new possibilities for studies on both brain neural activity decoding and music information retrieval.
面向音乐的听觉注意力检测（AAD）旨在通过分析听众的脑电图（EEG）来确定听众在复调音乐中关注的是哪种乐器。然而，现有的线性模型无法有效模拟人脑的非线性，导致性能有限。因此，本文提出了一种面向音乐的非线性AAD模型。首先，融合听觉特征和音乐特征，以精确、全面地表示音乐声源。其次，如果以立体声呈现音乐刺激，则增强脑电图。第三，构建神经网络架构，以捕捉脑电图与听觉刺激之间的非线性和动态相互作用。最后，在公共嵌入空间中与脑电图最相似的音乐声源被确定为被关注的声源。实验结果表明，所提出的模型优于所有基线模型。在1秒的决策窗口上，在单声道、双声道和三声道刺激下，其准确率分别达到92.6%和81.7%。此外，它可以很容易地扩展到面向语音的AAD。这项工作可以为脑神经元活动解码和音乐信息检索的研究开辟新的可能性。
