Department of Radiology, Mayo Clinic, Rochester, MN 55905, USA.
Department of Computer Science & Engineering, The University of Minnesota, Minneapolis, MN 55455, USA.
Tomography. 2022 Mar 24;8(2):905-919. doi: 10.3390/tomography8020073.
There is a growing demand for high-resolution (HR) medical images for both clinical and research applications. Image quality is inevitably traded off with acquisition time, which in turn impacts patient comfort, examination costs, dose, and motion-induced artifacts. For many image-based tasks, increasing the apparent spatial resolution in the perpendicular plane to produce multi-planar reformats or 3D images is commonly used. Single-image super-resolution (SR) is a promising technique to provide HR images based on deep learning to increase the resolution of a 2D image, but there are few reports on 3D SR. Further, perceptual loss is proposed in the literature to better capture the textural details and edges versus pixel-wise loss functions, by comparing the semantic distances in the high-dimensional feature space of a pre-trained 2D network (e.g., VGG). However, it is not clear how one should generalize it to 3D medical images, and the attendant implications are unclear. In this paper, we propose a framework called SOUP-GAN: uper-resolution ptimized sing erceptual-tuned enerative dversarial etwork (GAN), in order to produce thinner slices (e.g., higher resolution in the 'Z' plane) with anti-aliasing and deblurring. The proposed method outperforms other conventional resolution-enhancement methods and previous SR work on medical images based on both qualitative and quantitative comparisons. Moreover, we examine the model in terms of its generalization for arbitrarily user-selected SR ratios and imaging modalities. Our model shows promise as a novel 3D SR interpolation technique, providing potential applications for both clinical and research applications.
人们对用于临床和研究应用的高分辨率 (HR) 医学图像的需求日益增长。图像质量不可避免地要与采集时间进行权衡,这反过来又会影响患者的舒适度、检查成本、剂量和运动伪影。对于许多基于图像的任务,增加垂直平面上的表观空间分辨率以生成多平面重建或 3D 图像是常用的方法。基于深度学习的单图像超分辨率 (SR) 是一种很有前途的技术,可以提供 HR 图像,以提高二维图像的分辨率,但关于 3D SR 的报道很少。此外,文献中提出了感知损失,以通过比较预先训练的二维网络(例如 VGG)的高维特征空间中的语义距离,更好地捕获纹理细节和边缘,而不是逐像素损失函数。然而,尚不清楚如何将其推广到 3D 医学图像,其含义也不清楚。在本文中,我们提出了一个名为 SOUP-GAN 的框架:超分辨率优化感知调谐生成对抗网络(GAN),以产生具有抗混叠和去模糊效果的更薄切片(例如,“Z”平面上的更高分辨率)。与基于定性和定量比较的其他传统分辨率增强方法和以前的医学图像 SR 工作相比,所提出的方法表现出色。此外,我们还根据任意用户选择的 SR 比和成像模式来检查模型的泛化能力。我们的模型作为一种新颖的 3D SR 插值技术具有很大的应用潜力,为临床和研究应用提供了潜在的应用。