Shah Zafran Hussain, Müller Marcel, Hübner Wolfgang, Ortkrass Henning, Hammer Barbara, Huser Thomas, Schenck Wolfram
Center for Applied Data Science, Faculty of Engineering and Mathematics, Bielefeld University of Applied Sciences and Arts, Bielefeld, Germany.
Biomolecular Photonics Group, Faculty of Physics, Bielefeld University, Bielefeld, Germany.
Front Artif Intell. 2024 Sep 23;7:1353873. doi: 10.3389/frai.2024.1353873. eCollection 2024.
Real-valued convolutional neural networks (RV-CNNs) in the spatial domain have outperformed classical approaches in many image restoration tasks such as image denoising and super-resolution. Fourier analysis of the results produced by these spatial domain models reveals the limitations of these models in properly processing the full frequency spectrum. This lack of complete spectral information can result in missing textural and structural elements. To address this limitation, we explore the potential of complex-valued convolutional neural networks (CV-CNNs) for image restoration tasks. CV-CNNs have shown remarkable performance in tasks such as image classification and segmentation. However, CV-CNNs for image restoration problems in the frequency domain have not been fully investigated to address the aforementioned issues. Here, we propose several novel CV-CNN-based models equipped with complex-valued attention gates for image denoising and super-resolution in the frequency domains. We also show that our CV-CNN-based models outperform their real-valued counterparts for denoising super-resolution structured illumination microscopy (SR-SIM) and conventional image datasets. Furthermore, the experimental results show that our proposed CV-CNN-based models preserve the frequency spectrum better than their real-valued counterparts in the denoising task. Based on these findings, we conclude that CV-CNN-based methods provide a plausible and beneficial deep learning approach for image restoration in the frequency domain.
空间域中的实值卷积神经网络（RV-CNN）在许多图像恢复任务（如图像去噪和超分辨率）中表现优于传统方法。对这些空间域模型产生的结果进行傅里叶分析，揭示了这些模型在正确处理全频谱方面的局限性。这种完整频谱信息的缺乏可能导致纹理和结构元素的缺失。为了解决这一局限性，我们探索了复值卷积神经网络（CV-CNN）在图像恢复任务中的潜力。CV-CNN在图像分类和分割等任务中表现出了卓越的性能。然而，用于频域图像恢复问题的CV-CNN尚未得到充分研究以解决上述问题。在此，我们提出了几种基于CV-CNN的新型模型，这些模型配备了复值注意力门，用于频域中的图像去噪和超分辨率。我们还表明，我们基于CV-CNN的模型在去噪超分辨率结构光照显微镜（SR-SIM）和传统图像数据集方面优于实值对应模型。此外，实验结果表明，我们提出的基于CV-CNN的模型在去噪任务中比实值对应模型更好地保留了频谱。基于这些发现，我们得出结论，基于CV-CNN的方法为频域图像恢复提供了一种合理且有益的深度学习方法。
