Shi Xiaowen, Wang Yuan-Gen
Guangzhou University, School of Computer Science and Cyber Engineering, Guangzhou, 510006, China.
Sci Rep. 2024 Dec 28;14(1):31309. doi: 10.1038/s41598-024-82803-y.
Underwater image enhancement (UIE) is challenging since image degradation in aquatic environments is complicated and changing over time. Existing mainstream methods rely on either physical-model or data-driven, suffering from performance bottlenecks due to changes in imaging conditions or training instability. In this article, we attempt to adapt the diffusion model to the UIE task and propose a Content-Preserving Diffusion Model (CPDM) to address the above challenges. CPDM first leverages a diffusion model as its fundamental model for stable training and then designs a content-preserving framework to deal with changes in imaging conditions. Specifically, we construct a conditional input module by adopting both the raw image and the difference between the raw and noisy images as the input at each time step of the diffusion process, which can enhance the model's adaptability by considering the changes involving the raw images in underwater environments. To preserve the essential content of the raw images, we construct a content compensation module for content-aware training by extracting low-level image features of the raw images as compensation for each down block. We conducted tests on the LSUI, UIEB, and EUVP datasets, and the results show that CPDM outperforms state-of-the-art methods in both subjective and objective metrics, achieving the best overall performance. The GitHub link for the code is https://github.com/GZHU-DVL/CPDM.
水下图像增强(UIE)具有挑战性,因为水生环境中的图像退化复杂且随时间变化。现有的主流方法要么依赖物理模型,要么依赖数据驱动,由于成像条件的变化或训练的不稳定性而存在性能瓶颈。在本文中,我们尝试将扩散模型应用于UIE任务,并提出一种内容保留扩散模型(CPDM)来应对上述挑战。CPDM首先利用扩散模型作为其基础模型进行稳定训练,然后设计一个内容保留框架来处理成像条件的变化。具体来说,我们通过在扩散过程的每个时间步采用原始图像以及原始图像与噪声图像之间的差异作为输入来构建一个条件输入模块,这可以通过考虑水下环境中涉及原始图像的变化来提高模型的适应性。为了保留原始图像的基本内容,我们通过提取原始图像的低级图像特征作为每个下采样块的补偿来构建一个用于内容感知训练的内容补偿模块。我们在LSUI、UIEB和EUVP数据集上进行了测试,结果表明CPDM在主观和客观指标上均优于现有方法,实现了最佳的整体性能。代码的GitHub链接为https://github.com/GZHU-DVL/CPDM。