He Shuai, Zheng Shuntian, Ming Anlong, Wang Yanni, Ma Huadong
IEEE Trans Image Process. 2025;34:5300-5311. doi: 10.1109/TIP.2025.3594068.
The adage "Beautiful Outside But Ugly Inside" resonates with the security and explainability challenges encountered in image aesthetics assessment (IAA). Although deep neural networks (DNNs) have demonstrated remarkable performance in various IAA tasks, how to probe, explain, and enhance aesthetics-oriented "black-box" models has not yet been investigated to our knowledge. This lack of investigation has significantly impeded the commercial application of IAA. In this paper, we investigate the susceptibility of current IAA models to adversarial attacks and aim to elucidate the underlying mechanisms that contribute to their vulnerabilities. To address this, we propose a novel diffusion-based framework as an attacker (DA3Attacker), capable of generating adversarial examples (AEs) to deceive diverse black-box IAA models. DA3Attacker employs a dedicated Attack Diffusion Transformer, equipped with modular aesthetics-oriented filters. By undergoing two unsupervised training stages, it constructs a latent space to generate AEs and facilitates two distinct yet controllable attack modes: restricted and unrestricted. Extensive experiments on 26 baseline models demonstrate that our method effectively explores the vulnerabilities of these IAA models, while also providing multi-attribute explanations for their feature dependencies. To facilitate further research, we contribute the evaluation tools and four metrics for measuring adversarial robustness, as well as a dataset of 60,000 re-labeled AEs for fine-tuning IAA models. The resources are available here.
“金玉其外，败絮其中”这句格言与图像美学评估（IAA）中遇到的安全性和可解释性挑战相呼应。尽管深度神经网络（DNN）在各种IAA任务中都表现出了卓越的性能，但据我们所知，如何探究、解释和增强面向美学的“黑箱”模型尚未得到研究。这种研究的缺乏严重阻碍了IAA的商业应用。在本文中，我们研究了当前IAA模型对对抗攻击的敏感性，旨在阐明导致其脆弱性的潜在机制。为了解决这个问题，我们提出了一种新颖的基于扩散的框架作为攻击者（DA3Attacker），它能够生成对抗样本（AE）来欺骗各种黑箱IAA模型。DA3Attacker采用了一个专门的攻击扩散Transformer，配备了面向美学的模块化滤波器。通过经历两个无监督训练阶段，它构建了一个潜在空间来生成AE，并促进两种不同但可控的攻击模式：受限模式和无限制模式。对26个基线模型进行的大量实验表明，我们的方法有效地探索了这些IAA模型的脆弱性，同时还为它们的特征依赖关系提供了多属性解释。为了促进进一步的研究，我们贡献了评估工具和四个用于衡量对抗鲁棒性的指标，以及一个包含60000个重新标记的AE的数据集，用于微调IAA模型。相关资源可在此处获取。
