IEEE Trans Pattern Anal Mach Intell. 2022 Oct;44(10):6729-6751. doi: 10.1109/TPAMI.2021.3094362. Epub 2022 Sep 14.
Images can convey rich semantics and induce various emotions in viewers. Recently, with the rapid advancement of emotional intelligence and the explosive growth of visual data, extensive research efforts have been dedicated to affective image content analysis (AICA). In this survey, we will comprehensively review the development of AICA in the recent two decades, especially focusing on the state-of-the-art methods with respect to three main challenges - the affective gap, perception subjectivity, and label noise and absence. We begin with an introduction to the key emotion representation models that have been widely employed in AICA and description of available datasets for performing evaluation with quantitative comparison of label noise and dataset bias. We then summarize and compare the representative approaches on (1) emotion feature extraction, including both handcrafted and deep features, (2) learning methods on dominant emotion recognition, personalized emotion prediction, emotion distribution learning, and learning from noisy data or few labels, and (3) AICA based applications. Finally, we discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.
图像可以传达丰富的语义并引发观看者的各种情感。最近，随着情感智能的飞速发展和视觉数据的爆炸式增长，广泛的研究工作致力于情感图像内容分析（AICA）。在本次调查中，我们将全面回顾最近二十年来 AICA 的发展情况，特别是针对三个主要挑战——情感差距、感知主观性和标签噪声和缺失——的最新方法。我们首先介绍了在 AICA 中广泛使用的关键情感表示模型，并描述了可用的数据集，以便进行评估，并对标签噪声和数据集偏差进行定量比较。然后，我们总结并比较了代表性的方法，包括（1）情感特征提取，包括手工制作和深度特征，（2）主导情感识别、个性化情感预测、情感分布学习以及从噪声数据或少量标签学习的学习方法，以及（3）基于 AICA 的应用。最后，我们讨论了未来的一些挑战和有前途的研究方向，例如图像内容和上下文理解、群体情感聚类以及观看者与图像的交互。
