Buono Paolo, De Carolis Berardina, D'Errico Francesca, Macchiarulo Nicola, Palestra Giuseppe
Department of Computer Science, University of Bari 'Aldo Moro', Via Orabona 4, Bari, 70125 Italy.
Department Education, Psychology and Communication, University of Bari 'Aldo Moro', Via Crisanzio 42, Bari, 70122 Italy.
Multimed Tools Appl. 2023;82(9):12859-12877. doi: 10.1007/s11042-022-14048-8. Epub 2022 Oct 24.
The automatic monitoring and assessment of the engagement level of learners in distance education may help in understanding problems and providing personalized support during the learning process. This article presents a research aiming to investigate how student engagement level can be assessed from facial behavior and proposes a model based on Long Short-Term Memory (LSTM) networks to predict the level of engagement from facial action units, gaze, and head poses. The dataset used to learn the model is the one of the EmotiW 2019 challenge datasets. In order to test its performance in learning contexts, an experiment, involving students attending an online lecture, was performed. The aim of the study was to compare the self-evaluation of the engagement perceived by the students with the one assessed by the model. During the experiment we collected videos of students behavior and, at the end of each session, we asked students to answer a questionnaire for assessing their perceived engagement. Then, the collected videos were analyzed automatically with a software that implements the model and provides an interface for the visual analysis of the model outcome. Results show that, globally, engagement prediction from students' facial behavior was weakly correlated to their subjective answers. However, when considering only the emotional dimension of engagement, this correlation is stronger and the analysis of facial action units and head pose (facial movements) are positively correlated with it, while there is an inverse correlation with the gaze, meaning that the more the student's feels engaged the less are the gaze movements.
对远程教育中学习者参与度水平进行自动监测和评估，可能有助于了解问题并在学习过程中提供个性化支持。本文提出一项研究，旨在探究如何从面部行为评估学生的参与度水平，并提出一种基于长短期记忆（LSTM）网络的模型，用于根据面部动作单元、注视和头部姿势预测参与度水平。用于训练该模型的数据集是EmotiW 2019挑战赛数据集中的一个。为了测试其在学习环境中的性能，进行了一项涉及参加在线讲座的学生的实验。该研究的目的是比较学生对参与度的自我评估与模型评估的结果。在实验过程中，我们收集了学生行为的视频，并且在每节课结束时，我们要求学生回答一份问卷以评估他们感受到的参与度。然后，使用实现该模型并提供模型结果可视化分析界面的软件对收集到的视频进行自动分析。结果表明，总体而言，根据学生面部行为进行的参与度预测与他们的主观回答之间的相关性较弱。然而，仅考虑参与度的情感维度时，这种相关性更强，面部动作单元和头部姿势（面部运动）的分析与之呈正相关，而与注视呈负相关，这意味着学生感觉参与度越高，注视运动越少。
