Hubei Research Center for Educational Informationization, Faculty of Artificial Intelligence in Education, Central China Normal University, Wuhan 430074, China.
Huanggang High School of Hubei Province, Huanggang 438000, China.
Sensors (Basel). 2022 Aug 9;22(16):5932. doi: 10.3390/s22165932.
Engagement plays an essential role in the learning process. Recognition of learning engagement in the classroom helps us understand the student's learning state and optimize the teaching and study processes. Traditional recognition methods such as self-report and teacher observation are time-consuming and obtrusive to satisfy the needs of large-scale classrooms. With the development of big data analysis and artificial intelligence, applying intelligent methods such as deep learning to recognize learning engagement has become the research hotspot in education. In this paper, based on non-invasive classroom videos, first, a multi-cues classroom learning engagement database was constructed. Then, we introduced the power IoU loss function to You Only Look Once version 5 (YOLOv5) to detect the students and obtained a precision of 95.4%. Finally, we designed a bimodal learning engagement recognition method based on ResNet50 and CoAtNet. Our proposed bimodal learning engagement method obtained an accuracy of 93.94% using the KNN classifier. The experimental results confirmed that the proposed method outperforms most state-of-the-art techniques.
参与在学习过程中起着至关重要的作用。在课堂上识别学习参与度有助于我们了解学生的学习状态，并优化教学和学习过程。传统的识别方法，如自我报告和教师观察，既耗时又繁琐，难以满足大规模课堂的需求。随着大数据分析和人工智能的发展，应用深度学习等智能方法来识别学习参与度已成为教育领域的研究热点。在本文中，我们基于非侵入式课堂视频，首先构建了一个多线索课堂学习参与度数据库。然后，我们将 Power IoU 损失函数引入到 You Only Look Once 版本 5（YOLOv5）中，以检测学生，得到了 95.4%的精度。最后，我们设计了一种基于 ResNet50 和 CoAtNet 的双模态学习参与度识别方法。我们提出的双模态学习参与度方法使用 KNN 分类器获得了 93.94%的准确率。实验结果证实了该方法优于大多数最先进的技术。
