School of Electronics and Information Engineering, Tongji University, Shanghai, China.
School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, Texas, USA.
Med Phys. 2023 Jun;50(6):3560-3572. doi: 10.1002/mp.16144. Epub 2023 Jan 11.
Medical images have already become an essential tool for the diagnosis of many diseases. Thus a large number of medical images are being generated due to the daily routine inspection. An efficient image-based disease retrieval system will not only make full use of existing data, but also help physicians to prognosis the diseases. Medical image retrieval is represented by the classification and localization of common thorax diseases in x-ray images. Although extensive efforts have been put into this field, there are still many challenges.
Most of the existing fine-grained image research methods just apply existing deep learning frameworks in extracting the image features. However, these high-level features mainly focus on the global representations of the object, rather than simultaneously considering the local ones. It requires fine-grained details to classify the images with similar lesion areas. Thus, it is necessary to combine the global features and local ones to make the features more discriminative. On the other hand, training CNN models based on current existing strategies have a high time complexity, and is hard to get the discriminative features mentioned above. In addition, the visual retrieval method of fine-grained medical images still has the problem of insufficient sample data with accurate annotation information.
To address above challenges, we introduced a novel fine-grained medical images retrieval method. First, a centralized contrastive loss (CCLoss) is proposed as our metric learning loss function. Parameters are updated by using the center point, which not only improves the distinguishing performance of features, but also effectively reduces the time complexity of the algorithm. In addition, a weakly supervised progressive feature extraction method is proposed to gradually extract the combined features. And the attention mechanism module is applied to screen the target information after the initial positioning for fine refinement, so as to separate the features with a high degree of discrimination. The retrieval of 14 different chest diseases is evaluated on the chest x-ray datasets.
Compared with the existing research methods, the proposed method shows a better retrieval result for Recall@8 by 2.26 and achieves a very efficient training speed which is 100 times faster than the pair-wise loss-based training strategy. We also assessed the effects of Recall@k (k = 2, 4, 6, 8) for progressive features extracted from different steps to obtain a model with the best retrieval performance.
The proposed model is capable of learning discriminative representations from chest x-ray datasets, and it achieves better performance compared with other state-of-the-art methods. Therefore, the developed model would be useful in the diagnosis of common thorax disease or unknown chest disease.
医学图像已经成为许多疾病诊断的重要工具。因此，由于日常例行检查，大量的医学图像正在生成。高效的基于图像的疾病检索系统不仅可以充分利用现有数据，还可以帮助医生预测疾病。医学图像检索以 X 射线图像中常见胸部疾病的分类和定位为代表。尽管已经在这一领域进行了广泛的努力，但仍然存在许多挑战。
大多数现有的细粒度图像研究方法只是在提取图像特征时应用现有的深度学习框架。然而，这些高层特征主要关注对象的全局表示，而不是同时考虑局部表示。需要细粒度的细节来对具有相似病变区域的图像进行分类。因此，有必要结合全局特征和局部特征，使特征更具判别性。另一方面，基于当前现有策略训练 CNN 模型的时间复杂度较高，并且很难获得上述有判别力的特征。此外，细粒度医学图像的视觉检索方法仍然存在样本数据不足且标注信息准确的问题。
为了解决上述挑战，我们提出了一种新的细粒度医学图像检索方法。首先，提出了一种集中对比损失（CCLoss）作为我们的度量学习损失函数。通过使用中心点更新参数，不仅提高了特征的区分性能，而且有效地降低了算法的时间复杂度。此外，提出了一种弱监督渐进特征提取方法，用于逐步提取组合特征。并应用注意力机制模块对初始定位后的目标信息进行筛选，进行精细细化，分离具有高度判别力的特征。在胸部 X 射线数据集上评估了 14 种不同胸部疾病的检索。
与现有研究方法相比，该方法在 Recall@8 上的检索结果提高了 2.26%，并且实现了非常高效的训练速度，比基于成对损失的训练策略快 100 倍。我们还评估了从不同步骤提取的渐进特征的 Recall@k（k=2、4、6、8）的效果，以获得具有最佳检索性能的模型。
所提出的模型能够从胸部 X 射线数据集中学习有判别力的表示，并且与其他最先进的方法相比具有更好的性能。因此，该开发的模型将有助于常见胸部疾病或未知胸部疾病的诊断。
