Qian Qingfang, Chen Xiaofeng
Conservatory of Music, Qiongtai Normal University, Haikou 571100, China.
College of Teacher Education, Qiongtai Normal University, Haikou 571100, China.
Comput Intell Neurosci. 2022 Jul 19;2022:4957085. doi: 10.1155/2022/4957085. eCollection 2022.
The influence of music genres on children's emotional intelligence is one of the hot topics in the field of multi-modal emotion research. How to fuse multi-modal information has an important impact on children's emotional analysis. Most of the current research is based on transformer, in which the self-attention mechanism module is improved to achieve the fusion effect of multi-modal information. However, it is difficult for these methods to effectively capture the effective information of different modalities. Therefore, for the task of the influence of music genres on children's emotions, this paper proposes a transformer-based multi-modal convolutional neural network. The first is to use the BiLSTM sub-network model to extract the video and audio features and use the BERT sub-network to extract the text features. Secondly, this paper uses the improved transformer cross-modal fusion module to effectively fuse different types of modal information. Finally, the transformer module is used to judge the information of different modalities and analyze the emotion from the multi-modal information. At the same time, a large number of experiments prove that the model based on multi-modal convolutional neural network proposed in this paper surpasses other methods in prediction accuracy and effectively improves the accuracy of sentiment classification tasks.
音乐流派对儿童情商的影响是多模态情感研究领域的热点话题之一。如何融合多模态信息对儿童情感分析具有重要影响。当前的研究大多基于Transformer，其中对自注意力机制模块进行改进以实现多模态信息的融合效果。然而，这些方法难以有效捕捉不同模态的有效信息。因此，针对音乐流派对儿童情感影响的任务，本文提出了一种基于Transformer的多模态卷积神经网络。首先，使用双向长短期记忆（BiLSTM）子网络模型提取视频和音频特征，并使用BERT子网络提取文本特征。其次，本文使用改进的Transformer跨模态融合模块来有效融合不同类型的模态信息。最后，使用Transformer模块判断不同模态的信息并从多模态信息中分析情感。同时，大量实验证明，本文提出的基于多模态卷积神经网络的模型在预测准确率方面超越了其他方法，并有效提高了情感分类任务的准确率。
