Centre for Advanced Imaging, University of Queensland, Brisbane, Australia.
Centre for Advanced Imaging, University of Queensland, Brisbane, Australia; ARC Centre for Innovation in Biomedical Imaging Technology, Brisbane, Australia.
Comput Biol Med. 2022 Jul;146:105556. doi: 10.1016/j.compbiomed.2022.105556. Epub 2022 Apr 27.
Cross-modality image estimation involves the generation of images of one medical imaging modality from that of another modality. Convolutional neural networks (CNNs) have been shown to be useful in image-to-image intensity projections, in addition to identifying, characterising and extracting image patterns. Generative adversarial networks (GANs) use CNNs as generators and estimated images are classified as true or false based on an additional discriminator network. CNNs and GANs within the image estimation framework may be considered more generally as deep learning approaches, since medical images tend to be large in size, leading to the need for large neural networks. Most research in the CNN/GAN image estimation literature has involved the use of MRI data with the other modality primarily being PET or CT. This review provides an overview of the use of CNNs and GANs for cross-modality medical image estimation. We outline recently proposed neural networks and detail the constructs employed for CNN and GAN image-to-image synthesis. Motivations behind cross-modality image estimation are outlined as well. GANs appear to provide better utility in cross-modality image estimation in comparison with CNNs, a finding drawn based on our analysis involving metrics comparing estimated and actual images. Our final remarks highlight key challenges faced by the cross-modality medical image estimation field, including how intensity projection can be constrained by registration (unpaired versus paired data), use of image patches, additional networks, and spatially sensitive loss functions.
跨模态图像估计涉及从一种医学成像模式生成另一种模式的图像。卷积神经网络（CNN）已被证明在图像到图像的强度投影中很有用，除了识别、描述和提取图像模式之外。生成对抗网络（GAN）使用 CNN 作为生成器，根据附加的鉴别器网络将估计的图像分类为真实或虚假。图像估计框架中的 CNN 和 GAN 可以被更一般地视为深度学习方法，因为医学图像通常尺寸较大，因此需要大型神经网络。CNN/GAN 图像估计文献中的大多数研究都涉及使用 MRI 数据，而其他模态主要是 PET 或 CT。本文综述了 CNN 和 GAN 在跨模态医学图像估计中的应用。我们概述了最近提出的神经网络，并详细介绍了用于 CNN 和 GAN 图像到图像合成的结构。还概述了跨模态图像估计的动机。与 CNN 相比，GAN 似乎在跨模态图像估计中提供了更好的效用，这一发现是基于我们对比较估计图像和实际图像的指标的分析得出的。我们的最后评论强调了跨模态医学图像估计领域面临的关键挑战，包括如何通过配准（未配对与配对数据）、使用图像补丁、附加网络和空间敏感损失函数来约束强度投影。
