College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China.
College of Data Science, Taiyuan University of Technology, Taiyuan, Shanxi, China.
Comput Methods Programs Biomed. 2022 Oct;225:107106. doi: 10.1016/j.cmpb.2022.107106. Epub 2022 Sep 2.
Tiny spike-like projections on the basement membrane of glomeruli are the main pathological feature of membranous nephropathy at stage II (MN II), which is the most significant stage for the diagnosis and treatment of renal disease. Pathological technology is the gold standard in the diagnosis of spike-like and other MNs, and automatic classification of spike-like projection is a crucial step in assisting pathologists in their diagnosis. However, owing to hard-to-label spile-like projections and the scarcity of patient data, classification of glomeruli with spike-like projections based on supervised learning methods is a challenging task.
To overcome the aforementioned problems, the idea of integrating weakly-supervised learning and data augmentation methods is utilized in designing the classification framework. Specifically, a multiple instance learning with instance-level data augmentation (IDA-MIL) method for the classification of glomeruli with spike-like projections is established in this paper. The proposed classification framework first trains the MIL model on a dataset with image-level labels, and the well-trained MIL model is used to extract instances that include spike-like projections in the whole glomerular image. Then, rather than using an image-level generative adversarial network (ImgGAN), an instance-level generative adversarial network (InsGAN) based on the StyleGAN2-ADA model is trained on the spike-like instances obtained by the MIL model and synthesizes new spike-like projection instances. Finally, the synthesized spike-like instances are extended to the training dataset to further improve the classification performance of MIL.
The designed IDA-MIL model is verified and evaluated from two aspects based on the in-house dataset. On the one hand, the performance comparisons between InsGAN and ImgGAN on five metrics, which involve FID, KID, Precision, Recall and IS, show that InsGAN obtains a better score and can synthesize effective spike-like projections. However, the proposed IDA-MIL model achieves the best classification performance with an accuracy of 0.9405. Then, to make nephrologists believe the inference result of the proposed model, we use heatmap technology to visualize the basis of the model inferences and show the top 4 probability spike-like instances per glomerulus. Furthermore, we analyze the correlation between the disease and the proportion of spike-like instances in bags from historical cases.
Compared with the ImgGAN, the InsGAN can synthesize natural and varied spike-like projections, which results in the classification performance of the MIL model achieving great improvement by adding synthesized instance samples into the training dataset. The heatmap of spike-like inferences and the proportion of spike-like instances can help nephrologists to make a preliminary reliable diagnosis in clinical practice. This work provides a valuable reference for medical image classification with limited data and small-scale lesions based on deep learning.
肾小球基底膜上的微小刺状突起是膜性肾病二期（MN II）的主要病理特征，这是诊断和治疗肾脏疾病的最重要阶段。病理技术是诊断刺状突起和其他 MN 的金标准，而刺状突起的自动分类是协助病理学家诊断的关键步骤。然而，由于难以标记的刺状突起和患者数据的稀缺，基于监督学习方法对具有刺状突起的肾小球进行分类是一项具有挑战性的任务。
为了解决上述问题，该研究在设计分类框架时采用了弱监督学习和数据增强方法的思想。具体来说，本文建立了一种基于多实例学习和实例级数据增强（IDA-MIL）的具有刺状突起的肾小球分类方法。该分类框架首先在具有图像级标签的数据集上训练 MIL 模型，然后使用训练好的 MIL 模型提取整个肾小球图像中包含刺状突起的实例。然后，该研究不是使用图像级生成对抗网络（ImgGAN），而是基于 StyleGAN2-ADA 模型的实例级生成对抗网络（InsGAN）在 MIL 模型提取的刺状实例上进行训练，并合成新的刺状突起实例。最后，将合成的刺状实例扩展到训练数据集中，以进一步提高 MIL 的分类性能。
该研究基于内部数据集从两个方面验证和评估了设计的 IDA-MIL 模型。一方面，在 FID、KID、精度、召回率和 IS 等五个指标上对 InsGAN 和 ImgGAN 进行性能比较，结果表明 InsGAN 获得了更好的分数，并且可以合成有效的刺状突起。然而，所提出的 IDA-MIL 模型的分类性能最佳，准确率为 0.9405。然后，为了让肾病学家相信模型的推断结果，研究使用热图技术可视化模型推断的基础，并显示每个肾小球的前 4 个概率刺状实例。此外，研究还分析了历史病例中疾病与袋中刺状实例比例之间的相关性。
与 ImgGAN 相比，InsGAN 可以合成自然且多样的刺状突起，这使得 MIL 模型通过将合成的实例样本添加到训练数据集中，其分类性能得到了很大的提高。刺状推断的热图和刺状实例的比例可以帮助肾病学家在临床实践中做出初步可靠的诊断。这项工作为基于深度学习的有限数据和小病灶的医学图像分类提供了有价值的参考。
