Yi Yun, Zhou Yunkang, Wang Tinghua, Zhou Jin
School of Mathematics and Computer Science, Gannan Normal University, Ganzhou 341000, China.
Key Laboratory of Data Science and Artificial Intelligence of Jiangxi Education Institutes, Gannan Normal University, Ganzhou 341000, China.
Sensors (Basel). 2025 Jun 9;25(12):3615. doi: 10.3390/s25123615.
Video emotion recognition (VER), situated at the convergence of affective computing and computer vision, aims to predict the primary emotion evoked in most viewers through video content, with extensive applications in video recommendation, human-computer interaction, and intelligent education. This paper commences with an analysis of the psychological models that constitute the foundation of VER theory. The paper further elaborates on datasets and evaluation metrics commonly utilized in VER. Then, the paper reviews VER algorithms according to their categories, and compares and analyzes the experimental results of classic methods on four datasets. Based on a comprehensive analysis and investigations, the paper identifies the prevailing challenges currently faced in the VER field, including gaps between emotional representations and labels, large-scale and high-quality VER datasets, and the efficient integration of multiple modalities. Furthermore, this study proposes potential research directions to address these challenges, e.g., advanced neural network architectures, efficient multimodal fusion strategies, high-quality emotional representation, and robust active learning strategies.
视频情感识别（VER）处于情感计算和计算机视觉的交叉领域，旨在通过视频内容预测大多数观众所唤起的主要情感，在视频推荐、人机交互和智能教育等方面有广泛应用。本文首先分析构成VER理论基础的心理模型。接着详细阐述VER中常用的数据集和评估指标。然后，根据类别对VER算法进行综述，并比较和分析经典方法在四个数据集上的实验结果。基于全面的分析和调查，本文确定了VER领域目前面临的主要挑战，包括情感表征与标签之间的差距、大规模高质量的VER数据集以及多模态的有效整合。此外，本研究提出了应对这些挑战的潜在研究方向，例如先进的神经网络架构、高效的多模态融合策略、高质量的情感表征和强大的主动学习策略。
