Suppr超能文献

用于水下图像增强及其他领域的孪生对抗对比学习

Twin Adversarial Contrastive Learning for Underwater Image Enhancement and Beyond.

作者信息

Liu Risheng, Jiang Zhiying, Yang Shuzhou, Fan Xin

出版信息

IEEE Trans Image Process. 2022;31:4922-4936. doi: 10.1109/TIP.2022.3190209. Epub 2022 Jul 22.

Abstract

Underwater images suffer from severe distortion, which degrades the accuracy of object detection performed in an underwater environment. Existing underwater image enhancement algorithms focus on the restoration of contrast and scene reflection. In practice, the enhanced images may not benefit the effectiveness of detection and even lead to a severe performance drop. In this paper, we propose an object-guided twin adversarial contrastive learning based underwater enhancement method to achieve both visual-friendly and task-orientated enhancement. Concretely, we first develop a bilateral constrained closed-loop adversarial enhancement module, which eases the requirement of paired data with the unsupervised manner and preserves more informative features by coupling with the twin inverse mapping. In addition, to confer the restored images with a more realistic appearance, we also adopt the contrastive cues in the training phase. To narrow the gap between visually-oriented and detection-favorable target images, a task-aware feedback module is embedded in the enhancement process, where the coherent gradient information of the detector is incorporated to guide the enhancement towards the detection-pleasing direction. To validate the performance, we allocate a series of prolific detectors into our framework. Extensive experiments demonstrate that the enhanced results of our method show remarkable amelioration in visual quality, the accuracy of different detectors conducted on our enhanced images has been promoted notably. Moreover, we also conduct a study on semantic segmentation to illustrate how object guidance improves high-level tasks. Code and models are available at https://github.com/Jzy2017/TACL.

摘要

水下图像存在严重失真,这会降低在水下环境中进行目标检测的准确性。现有的水下图像增强算法主要集中在对比度恢复和场景反射方面。在实际应用中,增强后的图像可能对检测效果并无益处,甚至会导致性能严重下降。在本文中,我们提出了一种基于目标引导的双对抗对比学习的水下增强方法,以实现视觉友好型和任务导向型的增强效果。具体而言,我们首先开发了一个双边约束闭环对抗增强模块,该模块以无监督方式放宽了对成对数据的要求,并通过与双逆映射相结合来保留更多信息特征。此外,为了使恢复后的图像具有更逼真的外观,我们在训练阶段还采用了对比线索。为了缩小视觉导向型和检测友好型目标图像之间的差距,在增强过程中嵌入了一个任务感知反馈模块,其中纳入了检测器的相干梯度信息,以引导增强朝着有利于检测的方向进行。为了验证性能,我们在我们的框架中配置了一系列丰富的检测器。大量实验表明,我们方法的增强结果在视觉质量上有显著改善,在我们增强后的图像上进行的不同检测器的准确性也得到了显著提高。此外,我们还进行了语义分割研究,以说明目标引导如何改善高级任务。代码和模型可在https://github.com/Jzy2017/TACL获取。

文献AI研究员

20分钟写一篇综述,助力文献阅读效率提升50倍。

立即体验

用中文搜PubMed

大模型驱动的PubMed中文搜索引擎

马上搜索

文档翻译

学术文献翻译模型,支持多种主流文档格式。

立即体验