Liu Zhenguang, Wang Zepeng, Yao Yiyang, Zhang Luming, Shao Ling
IEEE Trans Image Process. 2018 Apr 18. doi: 10.1109/TIP.2018.2828326.
Image aesthetic quality assessment has becoming an indispensable technique that facilitates a variety of image applications, e.g., photo retargeting and non-realistic rendering. Conventional approaches suffer from the following limitations: 1) the inefficiency of semantically describing images due to the inherent tag noise and incompletion, 2) the difficulty of accurately reflecting how humans actively perceive various regions inside each image, and 3) the challenge of incorporating the aesthetic experiences of multiple users. To solve these problems, we propose a novel semi-supervised deep active learning (SDAL) algorithm, which discovers how humans perceive semantically important regions from a large quantity of images partially assigned with contaminated tags. More specifically, as humans usually attend to the foreground objects before understanding them, we extract a succinct set of BING (binarized normed gradients) [60]-based object patches from each image. To simulate human visual perception, we propose SDAL which hierarchically learns human gaze shifting path (GSP) by sequentially linking semantically important object patches from each scenery. Noticeably, SDLA unifies the semantically important regions discovery and deep GSP feature learning into a principled framework, wherein only a small proportion of tagged images are adopted. Moreover, based on the sparsity penalty, SDLA can optimally abandon the noisy or redundant low-level image features. Finally, by leveraging the deeply-learned GSP features, a probabilistic model is developed for image aesthetics assessment, where the experience of multiple professional photographers can be encoded. Besides, auxiliary quality-related features can be conveniently integrated into our probabilistic model. Comprehensive experiments on a series of benchmark image sets have demonstrated the superiority of our method. As a byproduct, eye tracking experiments have shown that GSPs generated by our SDAL are about 93% consistent with real human gaze shifting paths.
图像美学质量评估已成为一种不可或缺的技术，它推动了各种图像应用，例如照片重定目标尺寸和非真实感渲染。传统方法存在以下局限性：1）由于固有的标签噪声和不完整性，在语义上描述图像效率低下；2）难以准确反映人类如何主动感知每张图像内的各个区域；3）难以纳入多个用户的美学体验。为了解决这些问题，我们提出了一种新颖的半监督深度主动学习（SDAL）算法，该算法从大量带有受污染标签的部分图像中发现人类如何感知语义上重要的区域。更具体地说，由于人类通常在理解前景对象之前会先关注它们，我们从每张图像中提取了一组基于BING（二值化规范梯度）[60]的简洁对象块。为了模拟人类视觉感知，我们提出了SDAL，它通过依次链接每个场景中语义上重要的对象块，分层学习人类注视转移路径（GSP）。值得注意的是，SDLA将语义上重要区域的发现和深度GSP特征学习统一到一个有原则的框架中，其中仅采用一小部分带标签的图像。此外，基于稀疏性惩罚，SDLA可以最佳地舍弃噪声或冗余的低级图像特征。最后，通过利用深度学习的GSP特征，开发了一种用于图像美学评估的概率模型，其中可以编码多个专业摄影师的经验。此外，与质量相关的辅助特征可以方便地集成到我们的概率模型中。在一系列基准图像集上进行的综合实验证明了我们方法的优越性。作为一个副产品，眼动追踪实验表明，我们的SDAL生成的GSP与真实人类注视转移路径的一致性约为93%。
