Xie Huidong, Thorn Stephanie, Liu Yi-Hwa, Lee Supum, Liu Zhao, Wang Ge, Sinusas Albert J, Liu Chi
Department of Biomedical Engineering.
Department of Internal Medicine (Cardiology) at Yale University.
IEEE Trans Radiat Plasma Med Sci. 2023 Jan;7(1):33-40. doi: 10.1109/trpms.2022.3187595. Epub 2022 Jun 30.
Convolutional neural networks (CNNs) have been extremely successful in various medical imaging tasks. However, because the size of the convolutional kernel used in a CNN is much smaller than the image size, CNN has a strong spatial inductive bias and lacks a global understanding of the input images. Vision Transformer, a recently emerged network structure in computer vision, can potentially overcome the limitations of CNNs for image-reconstruction tasks. In this work, we proposed a slice-by-slice Transformer network (SSTrans-3D) to reconstruct cardiac SPECT images from 3D few-angle data. To be specific, the network reconstructs the whole 3D volume using a slice-by-slice scheme. By doing so, SSTrans-3D alleviates the memory burden required by 3D reconstructions using Transformer. The network can still obtain a global understanding of the image volume with the Transformer attention blocks. Lastly, already reconstructed slices are used as the input to the network so that SSTrans-3D can potentially obtain more informative features from these slices. Validated on porcine, phantom, and human studies acquired using a GE dedicated cardiac SPECT scanner, the proposed method produced images with clearer heart cavity, higher cardiac defect contrast, and more accurate quantitative measurements on the testing data as compared with a deep U-net.
卷积神经网络(CNN)在各种医学成像任务中都取得了巨大成功。然而,由于CNN中使用的卷积核尺寸远小于图像尺寸,CNN具有很强的空间归纳偏差,并且缺乏对输入图像的全局理解。视觉Transformer是计算机视觉中最近出现的一种网络结构,它有可能克服CNN在图像重建任务中的局限性。在这项工作中,我们提出了一种逐片Transformer网络(SSTrans-3D),用于从3D少角度数据重建心脏SPECT图像。具体来说,该网络采用逐片方案重建整个3D体积。通过这样做,SSTrans-3D减轻了使用Transformer进行3D重建所需的内存负担。该网络仍然可以通过Transformer注意力块获得对图像体积的全局理解。最后,已重建的切片用作网络的输入,以便SSTrans-3D可以潜在地从这些切片中获得更多信息丰富的特征。在使用GE专用心脏SPECT扫描仪获取的猪、体模和人体研究中进行验证,与深度U-net相比,该方法在测试数据上生成的图像具有更清晰的心脏腔、更高的心脏缺陷对比度和更准确的定量测量结果。