Wang Hao, Ahn Euijoon, Bi Lei, Kim Jinman
School of Computer Science, Faculty of Engineering, The University of Sydney, Sydney, NSW 2006, Australia; Institute of Translational Medicine, National Center for Translational Medicine, Shanghai Jiao Tong University, Shanghai, China.
College of Science and Engineering, James Cook University, Cairns, QLD 4870, Australia.
Comput Methods Programs Biomed. 2025 Jun;265:108729. doi: 10.1016/j.cmpb.2025.108729. Epub 2025 Apr 1.
The clinical diagnosis of skin lesions involves the analysis of dermoscopic and clinical modalities. Dermoscopic images provide detailed views of surface structures, while clinical images offer complementary macroscopic information. Clinicians frequently use the seven-point checklist as an auxiliary tool for melanoma diagnosis and identifying lesion attributes. Supervised deep learning approaches, such as convolutional neural networks, have performed well using dermoscopic and clinical modalities (multi-modality) and further enhanced classification by predicting seven skin lesion attributes (multi-label). However, the performance of these approaches is reliant on the availability of large-scale labeled data, which are costly and time-consuming to obtain, more so with annotating multi-attributes METHODS:: To reduce the dependency on large labeled datasets, we propose a self-supervised learning (SSL) algorithm for multi-modality multi-label skin lesion classification. Compared with single-modality SSL, our algorithm enables multi-modality SSL by maximizing the similarities between paired dermoscopic and clinical images from different views. We introduce a novel multi-modal and multi-label SSL strategy that generates surrogate pseudo-multi-labels for seven skin lesion attributes through clustering analysis. A label-relation-aware module is proposed to refine each pseudo-label embedding, capturing the interrelationships between pseudo-multi-labels. We further illustrate the interrelationships of skin lesion attributes and their relationships with clinical diagnoses using an attention visualization technique.
The proposed algorithm was validated using the well-benchmarked seven-point skin lesion dataset. Our results demonstrate that our method outperforms the state-of-the-art SSL counterparts. Improvements in the area under receiver operating characteristic curve, precision, sensitivity, and specificity were observed across various lesion attributes and melanoma diagnoses.
Our self-supervised learning algorithm offers a robust and efficient solution for multi-modality multi-label skin lesion classification, reducing the reliance on large-scale labeled data. By effectively capturing and leveraging the complementary information between the dermoscopic and clinical images and interrelationships between lesion attributes, our approach holds the potential for improving clinical diagnosis accuracy in dermatology.
皮肤病变的临床诊断涉及皮肤镜和临床模式的分析。皮肤镜图像提供表面结构的详细视图，而临床图像提供补充的宏观信息。临床医生经常使用七点检查表作为黑色素瘤诊断和识别病变特征的辅助工具。监督深度学习方法，如卷积神经网络，在使用皮肤镜和临床模式（多模式）方面表现良好，并通过预测七种皮肤病变特征（多标签）进一步增强分类。然而，这些方法的性能依赖于大规模标记数据的可用性，而获取这些数据成本高且耗时，对于多属性标注更是如此。
为了减少对大型标记数据集的依赖，我们提出了一种用于多模式多标签皮肤病变分类的自监督学习（SSL）算法。与单模式SSL相比，我们的算法通过最大化来自不同视图的配对皮肤镜和临床图像之间的相似性来实现多模式SSL。我们引入了一种新颖的多模态多标签SSL策略，通过聚类分析为七种皮肤病变特征生成替代伪多标签。提出了一个标签关系感知模块来细化每个伪标签嵌入，捕捉伪多标签之间的相互关系。我们进一步使用注意力可视化技术说明了皮肤病变特征的相互关系及其与临床诊断的关系。
使用经过充分基准测试的七点皮肤病变数据集对所提出的算法进行了验证。我们的结果表明，我们的方法优于现有的SSL同类方法。在各种病变特征和黑色素瘤诊断中，观察到接收器操作特征曲线下面积、精度、敏感性和特异性的提高。
我们的自监督学习算法为多模式多标签皮肤病变分类提供了一种强大而有效的解决方案，减少了对大规模标记数据的依赖。通过有效捕捉和利用皮肤镜和临床图像之间的互补信息以及病变特征之间的相互关系，我们的方法具有提高皮肤病临床诊断准确性的潜力。
