School of Integrated Circuits, Anhui University, Hefei 230601, China.
AnHui Engineering Laboratory of Agro-Ecological Big Data, Hefei 230601, China.
Sensors (Basel). 2022 Jul 21;22(14):5464. doi: 10.3390/s22145464.
In recent years, deep convolutional neural network (CNN)-based image enhancement has shown outstanding performance. However, due to the problems of uneven illumination and low contrast existing in endoscopic images, the implementation of medical endoscopic image enhancement using CNN is still an exploratory and challenging task. An endoscopic image enhancement network (EIEN) based on the Retinex theory is proposed in this paper to solve these problems. The structure consists of three parts: decomposition network, illumination correction network, and reflection component enhancement algorithm. First, the decomposition network model of pre-trained Retinex-Net is retrained on the endoscopic image dataset, and then the images are decomposed into illumination and reflection components by this decomposition network. Second, the illumination components are corrected by the proposed self-attention guided multi-scale pyramid structure. The pyramid structure is used to capture the multi-scale information of the image. The self-attention mechanism is based on the imaging nature of the endoscopic image, and the inverse image of the illumination component is fused with the features of the green and blue channels of the image to be enhanced to generate a weight map that reassigns weights to the spatial dimension of the feature map, to avoid the loss of details in the process of multi-scale feature fusion and image reconstruction by the network. The reflection component enhancement is achieved by sub-channel stretching and weighted fusion, which is used to enhance the vascular information and image contrast. Finally, the enhanced illumination and reflection components are multiplied to obtain the reconstructed image. We compare the results of the proposed method with six other methods on a test set. The experimental results show that EIEN enhances the brightness and contrast of endoscopic images and highlights vascular and tissue information. At the same time, the method in this paper obtained the best results in terms of visual perception and objective evaluation.
近年来，基于深度卷积神经网络（CNN）的图像增强技术表现出色。然而，由于内窥镜图像存在光照不均匀和对比度低的问题，使用 CNN 实现医学内窥镜图像增强仍然是一个探索性和具有挑战性的任务。本文提出了一种基于 Retinex 理论的内窥镜图像增强网络（EIEN），旨在解决这些问题。该网络结构由三部分组成：分解网络、光照校正网络和反射分量增强算法。首先，在内窥镜图像数据集上重新训练预先训练的 Retinex-Net 分解网络模型，然后通过该分解网络将图像分解为光照分量和反射分量。其次，通过所提出的自注意力引导多尺度金字塔结构校正光照分量。金字塔结构用于捕获图像的多尺度信息。自注意力机制基于内窥镜图像的成像特性，将光照分量的逆图像与要增强的图像的绿色通道和蓝色通道的特征融合，生成一个权重图，重新分配特征图的空间维度的权重，以避免在多尺度特征融合和网络图像重建过程中丢失细节。通过子通道拉伸和加权融合来实现反射分量的增强，用于增强血管信息和图像对比度。最后，将增强的光照和反射分量相乘得到重建图像。我们在测试集上将提出的方法与其他六种方法的结果进行了比较。实验结果表明，EIEN 增强了内窥镜图像的亮度和对比度，突出了血管和组织信息。同时，本文提出的方法在视觉感知和客观评价方面取得了最佳效果。
