Jiangsu Key Laboratory for Biomaterials and Devices, School of Biological Science and Medical Engineering, Southeast University, Nanjing, China.
Med Phys. 2024 Nov;51(11):8283-8301. doi: 10.1002/mp.17362. Epub 2024 Aug 17.
The current mainstream multi-modal medical image-to-image translation methods face a contradiction. Supervised methods with outstanding performance rely on pixel-wise aligned training data to constrain the model optimization. However, obtaining pixel-wise aligned multi-modal medical image datasets is challenging. Unsupervised methods can be trained without paired data, but their reliability cannot be guaranteed. At present, there is no ideal multi-modal medical image-to-image translation method that can generate reliable translation results without the need for pixel-wise aligned data.
This work aims to develop a novel medical image-to-image translation model that is independent of pixel-wise aligned data (MITIA), enabling reliable multi-modal medical image-to-image translation under the condition of misaligned training data.
The proposed MITIA model utilizes a prior extraction network composed of a multi-modal medical image registration module and a multi-modal misalignment error detection module to extract pixel-level prior information from training data with misalignment errors to the largest extent. The extracted prior information is then used to construct a regularization term to constrain the optimization of the unsupervised cycle-consistent Generative Adversarial Network model, restricting its solution space and thereby improving the performance and reliability of the generator. We trained the MITIA model using six datasets containing different misalignment errors and two well-aligned datasets. Subsequently, we conducted quantitative analysis using peak signal-to-noise ratio and structural similarity as metrics. Moreover, we compared the proposed method with six other state-of-the-art image-to-image translation methods.
The results of both quantitative analysis and qualitative visual inspection indicate that MITIA achieves superior performance compared to the competing state-of-the-art methods, both on misaligned data and aligned data. Furthermore, MITIA shows more stability in the presence of misalignment errors in the training data, regardless of their severity or type.
The proposed method achieves outstanding performance in multi-modal medical image-to-image translation tasks without aligned training data. Due to the difficulty in obtaining pixel-wise aligned data for medical image translation tasks, MITIA is expected to generate significant application value in this scenario compared to existing methods.
当前主流的多模态医学图像到图像翻译方法面临着一个矛盾。表现出色的监督方法依赖于像素对齐的训练数据来约束模型优化。然而，获得像素对齐的多模态医学图像数据集具有挑战性。无监督方法可以在没有配对数据的情况下进行训练，但它们的可靠性无法保证。目前，没有理想的多模态医学图像到图像翻译方法可以在不需要像素对齐数据的情况下生成可靠的翻译结果。
本研究旨在开发一种新的医学图像到图像翻译模型，该模型不依赖于像素对齐数据（MITIA），能够在训练数据存在未对齐的情况下实现可靠的多模态医学图像到图像翻译。
所提出的 MITIA 模型利用了一个由多模态医学图像配准模块和多模态未对齐误差检测模块组成的先验提取网络，从存在未对齐误差的训练数据中最大限度地提取像素级先验信息。然后，将提取的先验信息用于构建正则化项，以约束无监督循环一致性生成对抗网络模型的优化，限制其解空间，从而提高生成器的性能和可靠性。我们使用包含不同未对齐误差的六个数据集和两个对齐良好的数据集来训练 MITIA 模型。随后，我们使用峰值信噪比和结构相似性作为指标进行定量分析。此外，我们将所提出的方法与其他六种最新的图像到图像翻译方法进行了比较。
定量分析和定性视觉检查的结果均表明，MITIA 在存在未对齐数据和对齐数据的情况下，与竞争的最先进方法相比，性能更优。此外，MITIA 在训练数据中存在未对齐误差时表现出更高的稳定性，无论误差的严重程度或类型如何。
所提出的方法在没有对齐训练数据的情况下，在多模态医学图像到图像翻译任务中取得了出色的性能。由于医学图像翻译任务中获取像素对齐数据的困难，与现有方法相比，MITIA 在这种情况下有望产生显著的应用价值。
