Yu Miao, Guo Miaomiao, Zhang Shuai, Zhan Yuefu, Zhao Mingkang, Lukasiewicz Thomas, Xu Zhenghua
State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin, China.
State Key Laboratory of Reliability and Intelligence of Electrical Equipment, School of Health Sciences and Biomedical Engineering, Hebei University of Technology, Tianjin, China.
Comput Biol Med. 2023 Oct 27;167:107632. doi: 10.1016/j.compbiomed.2023.107632.
A common problem in the field of deep-learning-based low-level vision medical images is that most of the research is based on single task learning (STL), which is dedicated to solving one of the situations of low resolution or high noise. Our motivation is to design a model that can perform both SR and DN tasks simultaneously, in order to cope with the actual situation of low resolution and high noise in low-level vision medical images. By improving the existing single image super-resolution (SISR) network and introducing the idea of multi-task learning (MTL), we propose an end-to-end lightweight MTL generative adversarial network (GAN) based network using residual-in-residual-blocks (RIR-Blocks) for feature extraction, RIRGAN, which can concurrently accomplish super-resolution (SR) and denoising (DN) tasks. The generator in RIRGAN is composed of several residual groups with a long skip connection (LSC), which can help form a very deep network and enable the network to focus on learning high-frequency (HF) information. The introduction of a discriminator based on relativistic average discriminator (RaD) greatly improves the discriminator's ability and makes the generated image have more realistic details. Meanwhile, the use of hybrid loss function not only ensures that RIRGAN has the ability of MTL, but also enables RIRGAN to give a more balanced attention between quantitative evaluation of metrics and qualitative evaluation of human vision. The experimental results show that the quality of the restoration image of RIRGAN is superior to the SR and DN methods based on STL in both subjective perception and objective evaluation metrics when processing medical images with low-level vision. Our RIRGAN is more in line with the practical requirements of medical practice.
基于深度学习的低层次视觉医学图像领域中的一个常见问题是，大多数研究基于单任务学习（STL），该学习致力于解决低分辨率或高噪声情况中的一种。我们的动机是设计一个能够同时执行超分辨率（SR）和去噪（DN）任务的模型，以应对低层次视觉医学图像中低分辨率和高噪声的实际情况。通过改进现有的单图像超分辨率（SISR）网络并引入多任务学习（MTL）的思想，我们提出了一种基于残差块（RIR-Blocks）进行特征提取的端到端轻量级MTL生成对抗网络（GAN），即RIRGAN，它可以同时完成超分辨率（SR）和去噪（DN）任务。RIRGAN中的生成器由几个带有长跳跃连接（LSC）的残差组组成，这有助于形成一个非常深的网络，并使网络能够专注于学习高频（HF）信息。基于相对论平均判别器（RaD）的判别器的引入大大提高了判别器的能力，并使生成的图像具有更逼真的细节。同时，混合损失函数的使用不仅确保RIRGAN具有MTL的能力，还使RIRGAN能够在指标的定量评估和人类视觉的定性评估之间给予更平衡的关注。实验结果表明，在处理低层次视觉医学图像时，RIRGAN的恢复图像质量在主观感知和客观评估指标方面均优于基于STL的SR和DN方法。我们的RIRGAN更符合医学实践的实际要求。
