IEEE Trans Image Process. 2017 Mar;26(3):1482-1495. doi: 10.1109/TIP.2017.2651399. Epub 2017 Jan 11.
Human beings often assess the aesthetic quality of an image coupled with the identification of the image's semantic content. This paper addresses the correlation issue between automatic aesthetic quality assessment and semantic recognition. We cast the assessment problem as the main task among a multi-task deep model, and argue that semantic recognition task offers the key to address this problem. Based on convolutional neural networks, we employ a single and simple multi-task framework to efficiently utilize the supervision of aesthetic and semantic labels. A correlation item between these two tasks is further introduced to the framework by incorporating the inter-task relationship learning. This item not only provides some useful insight about the correlation but also improves assessment accuracy of the aesthetic task. In particular, an effective strategy is developed to keep a balance between the two tasks, which facilitates to optimize the parameters of the framework. Extensive experiments on the challenging Aesthetic Visual Analysis dataset and Photo.net dataset validate the importance of semantic recognition in aesthetic quality assessment, and demonstrate that multitask deep models can discover an effective aesthetic representation to achieve the state-of-the-art results.
人类经常会评估图像的美学质量，并识别图像的语义内容。本文探讨了自动美学质量评估和语义识别之间的关联问题。我们将评估问题作为多任务深度学习模型中的主要任务，并认为语义识别任务是解决该问题的关键。基于卷积神经网络，我们采用单一而简单的多任务框架，有效地利用美学和语义标签的监督信息。通过引入任务间的关系学习，进一步在框架中引入了这两个任务之间的相关项。该相关项不仅提供了一些关于相关性的有用见解，而且还提高了美学任务的评估准确性。特别是，我们开发了一种有效的策略来平衡这两个任务，这有助于优化框架的参数。在具有挑战性的美学视觉分析数据集和 Photo.net 数据集上的广泛实验验证了语义识别在美学质量评估中的重要性，并表明多任务深度学习模型可以发现有效的美学表示，从而实现最先进的结果。
