Shen Yumin, Guo Hongyu
School of Foreign Languages, Zhejiang Gongshang University, Hangzhou, China.
Graduate School of Education, University of Perpetual Help System DALTA, Metro Manila, Philippinesa.
Front Psychol. 2022 Feb 11;13:839440. doi: 10.3389/fpsyg.2022.839440. eCollection 2022.
The outbreak of COVID-19 has brought drastic changes to English teaching as it has shifted from the offline mode before the pandemic to the online mode during the pandemic. However, in the post-pandemic era, there are still many problems in the effective implementation of the process of English teaching, leading to the inability of achieving better results in the quality and efficiency of English teaching and effective cultivation of students' practical application ability. In recent years, English speaking has attracted the attention of experts and scholars. Therefore, this study constructs an interactive English-speaking practice scene based on a virtual character. A dual-modality emotion recognition method is proposed that mainly recognizes and analyzes facial expressions and physiological signals of students and the virtual character in each scene. Thereafter, the system adjusts the difficulty of the conversation according to the current state of students, toward making the conversation more conducive to the students' understanding and gradually improving their English-speaking ability. The simulation compares nine facial expressions based on the eNTERFACE05 and CAS-PEAL datasets, which shows that the emotion recognition method proposed in this manuscript can effectively recognize students' emotions in interactive English-speaking practice and reduce the recognition time to a great extent. The recognition accuracy of the nine facial expressions was close to 90% for the dual-modality emotion recognition method in the eNTERFACE05 dataset, and the recognition accuracy of the dual-modality emotion recognition method was significantly improved with an average improvement of approximately 5%.
新冠疫情的爆发给英语教学带来了巨大变化,英语教学从疫情前的线下模式转变为疫情期间的线上模式。然而,在后疫情时代,英语教学过程的有效实施仍存在诸多问题,导致英语教学质量和效率无法取得更好的成果,也无法有效培养学生的实际应用能力。近年来,英语口语受到了专家学者的关注。因此,本研究构建了一个基于虚拟角色的交互式英语口语练习场景。提出了一种双模态情感识别方法,主要用于识别和分析每个场景中学生和虚拟角色的面部表情及生理信号。此后,系统根据学生的当前状态调整对话难度,以使对话更有利于学生理解,逐步提高他们的英语口语能力。该模拟基于eNTERFACE05和CAS-PEAL数据集比较了九种面部表情,结果表明本文提出的情感识别方法能够在交互式英语口语练习中有效识别学生的情绪,并在很大程度上缩短识别时间。在eNTERFACE05数据集中,双模态情感识别方法对九种面部表情的识别准确率接近90%,且双模态情感识别方法的识别准确率有显著提高,平均提高了约5%。