Super-resolution of 3D medical images by generative adversarial networks with long and short-term memory and attention.

Since 3D medical imaging data is a string of sequential images, there is a strong correlation between consecutive images. Deep convolutional networks perform well in extracting spatial features, but are less capable for processing sequence data compared to recurrent convolutional networks. Therefore, we propose a long short-term memory and attention based generative adversarial network (LSTMAGAN) to realize super-resolution reconstruction of 3D medical image. Firstly, we use generative adversarial networks as the base model for super-resolution image reconstruction. Secondly, a long and short-term memory network, which specializes in dealing with long-term dependencies in sequential data, was used to process continuous sequential data of 3D medical images based on its ability to remember and forget information efficiently. Next, an attention gate is used to suppress the background noise information and improve the clarity of image features. Finally, the method proposed in this paper is applied on the Luna16 and BraTs2021 datasets. The experimental results show that the proposed method improves the PSNR and SSIM evaluation indexes compared with other comparative methods, respectively. Therefore, it can prove the advancement and effectiveness of the proposed method.