Yan Siyuan, Yu Zhen, Primiero Clare, Vico-Alonso Cristina, Wang Zhonghua, Yang Litao, Tschandl Philipp, Hu Ming, Ju Lie, Tan Gin, Tang Vincent, Ng Aik Beng, Powell David, Bonnington Paul, See Simon, Magnaterra Elisabetta, Ferguson Peter, Nguyen Jennifer, Guitera Pascale, Banuls Jose, Janda Monika, Mar Victoria, Kittler Harald, Soyer H Peter, Ge Zongyuan
AIM for Health Lab, Faculty of Information Technology, Monash University, Melbourne, Victoria, Australia.
Faculty of Engineering, Monash University, Melbourne, Victoria, Australia.
Nat Med. 2025 Jun 6. doi: 10.1038/s41591-025-03747-y.
Diagnosing and treating skin diseases require advanced visual skills across domains and the ability to synthesize information from multiple imaging modalities. While current deep learning models excel at specific tasks such as skin cancer diagnosis from dermoscopic images, they struggle to meet the complex, multimodal requirements of clinical practice. Here we introduce PanDerm, a multimodal dermatology foundation model pretrained through self-supervised learning on over 2 million real-world skin disease images from 11 clinical institutions across 4 imaging modalities. We evaluated PanDerm on 28 diverse benchmarks, including skin cancer screening, risk stratification, differential diagnosis of common and rare skin conditions, lesion segmentation, longitudinal monitoring, and metastasis prediction and prognosis. PanDerm achieved state-of-the-art performance across all evaluated tasks, often outperforming existing models when using only 10% of labeled data. We conducted three reader studies to assess PanDerm's potential clinical utility. PanDerm outperformed clinicians by 10.2% in early-stage melanoma detection through longitudinal analysis, improved clinicians' skin cancer diagnostic accuracy by 11% on dermoscopy images and enhanced nondermatologist healthcare providers' differential diagnosis by 16.5% across 128 skin conditions on clinical photographs. These results show PanDerm's potential to improve patient care across diverse clinical scenarios and serve as a model for developing multimodal foundation models in other medical specialties, potentially accelerating the integration of artificial intelligence support in healthcare.
诊断和治疗皮肤疾病需要跨领域的先进视觉技能以及综合多种成像方式信息的能力。虽然当前的深度学习模型在诸如从皮肤镜图像诊断皮肤癌等特定任务上表现出色，但它们难以满足临床实践中复杂的多模态要求。在此，我们引入了PanDerm，这是一种多模态皮肤病学基础模型，通过对来自4种成像方式的11个临床机构的200多万张真实世界皮肤疾病图像进行自监督学习进行预训练。我们在28个不同的基准上对PanDerm进行了评估，包括皮肤癌筛查、风险分层、常见和罕见皮肤疾病的鉴别诊断、病变分割、纵向监测以及转移预测和预后。PanDerm在所有评估任务中都取得了领先的性能，在仅使用10%的标记数据时，其表现往往优于现有模型。我们进行了三项读者研究以评估PanDerm的潜在临床效用。通过纵向分析，PanDerm在早期黑色素瘤检测方面比临床医生表现优10.2%，在皮肤镜图像上提高了临床医生11%的皮肤癌诊断准确率，在临床照片上针对128种皮肤疾病提高了非皮肤科医疗服务提供者16.5%的鉴别诊断能力。这些结果表明，PanDerm有潜力在各种临床场景中改善患者护理，并成为其他医学专业开发多模态基础模型的典范，有可能加速人工智能支持在医疗保健中的整合。
