Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, 250358, China.
Shandong Key Laboratory of Medical Physics and Image Processing, Shandong Institute of Industrial Technology for Health Sciences and Precision Medicine, School of Physics and Electronics, Shandong Normal University, Jinan, 250358, China.
Med Image Anal. 2020 Jul;63:101667. doi: 10.1016/j.media.2020.101667. Epub 2020 Apr 22.
Contrast-enhanced magnetic resonance imaging (CEMRI) is crucial for the diagnosis of patients with liver tumors, especially for the detection of benign tumors and malignant tumors. However, it suffers from high-risk, time-consuming, and expensive in current clinical diagnosis due to the use of the gadolinium-based contrast agent (CA) injection. If the CEMRI can be synthesized without CA injection, there is no doubt that it will greatly optimize the diagnosis. In this study, we propose a Tripartite Generative Adversarial Network (Tripartite-GAN) as a non-invasive, time-saving, and inexpensive clinical tool by synthesizing CEMRI to detect tumors without CA injection. Specifically, our innovative Tripartite-GAN combines three associated-networks (an attention-aware generator, a convolutional neural network-based discriminator, and a region-based convolutional neural network-based detector) for the first time, which achieves CEMRI synthesis and tumor detection promoting each other in an end-to-end framework. The generator facilitates detector for accurate tumor detection via synthesizing tumor-specific CEMRI. The detector promotes the generator for accurate CEMRI synthesis via the back-propagation. In order to synthesize CEMRI of equivalent clinical value to real CEMRI, the attention-aware generator expands the receptive field via hybrid convolution, and enhances feature representation and context learning of multi-class liver MRI via dual attention mechanism, and improves the performance of convergence of loss via residual learning. Moreover, the attention maps obtained from the generator newly added into the detector improve the performance of tumor detection. The discriminator promotes the generator to synthesize high-quality CEMRI via the adversarial learning strategy. This framework is tested on a large corpus of axial T1 FS Pre-Contrast MRI and axial T1 FS Delay MRI of 265 subjects. Experimental results and quantitative evaluation demonstrate that the Tripartite-GAN achieves high-quality CEMRI synthesis that peak signal-to-noise rate of 28.8 and accurate tumor detection that accuracy of 89.4%, which reveals that Tripartite-GAN can aid in the clinical diagnosis of liver tumors.
对比增强磁共振成像（CEMRI）对于肝脏肿瘤患者的诊断至关重要，特别是对良性肿瘤和恶性肿瘤的检测。然而，由于使用钆基造影剂（CA）注射，目前的临床诊断存在风险高、耗时和昂贵的问题。如果可以不注射 CA 就合成 CEMRI，无疑将极大地优化诊断。在这项研究中，我们提出了一种三方生成对抗网络（Tripartite-GAN），作为一种非侵入性、节省时间和成本低廉的临床工具，通过不注射 CA 就合成 CEMRI 来检测肿瘤。具体来说，我们的创新型三方 GAN 首次结合了三个关联网络（一个注意感知生成器、一个基于卷积神经网络的鉴别器和一个基于区域的卷积神经网络的检测器），在端到端框架中实现了 CEMRI 合成和肿瘤检测的相互促进。生成器通过合成肿瘤特异性的 CEMRI 来帮助检测器进行准确的肿瘤检测。检测器通过反向传播来促进生成器进行准确的 CEMRI 合成。为了合成具有与真实 CEMRI 等效临床价值的 CEMRI，注意感知生成器通过混合卷积扩展了感受野，并通过双注意力机制增强了多类肝脏 MRI 的特征表示和上下文学习，通过残差学习提高了损失的收敛性能。此外，从生成器中获得的注意力图新添加到检测器中，提高了肿瘤检测的性能。鉴别器通过对抗学习策略促进生成器合成高质量的 CEMRI。该框架在 265 名受试者的大量轴向 T1 FS 预对比 MRI 和轴向 T1 FS 延迟 MRI 数据上进行了测试。实验结果和定量评估表明，三方 GAN 实现了高质量的 CEMRI 合成，峰值信噪比为 28.8，准确的肿瘤检测，准确率为 89.4%，这表明三方 GAN 可以辅助肝脏肿瘤的临床诊断。
