Digital Biomarkers for Oncology Group (DBO), National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.
Department of Medicine III, RWTH University Hospital Aachen, Aachen, Germany.
J Med Internet Res. 2021 Jul 2;23(7):e20708. doi: 10.2196/20708.
Recent years have been witnessing a substantial improvement in the accuracy of skin cancer classification using convolutional neural networks (CNNs). CNNs perform on par with or better than dermatologists with respect to the classification tasks of single images. However, in clinical practice, dermatologists also use other patient data beyond the visual aspects present in a digitized image, further increasing their diagnostic accuracy. Several pilot studies have recently investigated the effects of integrating different subtypes of patient data into CNN-based skin cancer classifiers.
This systematic review focuses on the current research investigating the impact of merging information from image features and patient data on the performance of CNN-based skin cancer image classification. This study aims to explore the potential in this field of research by evaluating the types of patient data used, the ways in which the nonimage data are encoded and merged with the image features, and the impact of the integration on the classifier performance.
Google Scholar, PubMed, MEDLINE, and ScienceDirect were screened for peer-reviewed studies published in English that dealt with the integration of patient data within a CNN-based skin cancer classification. The search terms skin cancer classification, convolutional neural network(s), deep learning, lesions, melanoma, metadata, clinical information, and patient data were combined.
A total of 11 publications fulfilled the inclusion criteria. All of them reported an overall improvement in different skin lesion classification tasks with patient data integration. The most commonly used patient data were age, sex, and lesion location. The patient data were mostly one-hot encoded. There were differences in the complexity that the encoded patient data were processed with regarding deep learning methods before and after fusing them with the image features for a combined classifier.
This study indicates the potential benefits of integrating patient data into CNN-based diagnostic algorithms. However, how exactly the individual patient data enhance classification performance, especially in the case of multiclass classification problems, is still unclear. Moreover, a substantial fraction of patient data used by dermatologists remains to be analyzed in the context of CNN-based skin cancer classification. Further exploratory analyses in this promising field may optimize patient data integration into CNN-based skin cancer diagnostics for patients' benefits.
近年来，卷积神经网络（CNN）在皮肤癌分类的准确性方面取得了实质性的提高。在单张图像的分类任务上，CNN 的表现与皮肤科医生相当，甚至优于皮肤科医生。然而，在临床实践中，皮肤科医生还会使用除了数字化图像中呈现的视觉方面之外的其他患者数据，从而进一步提高他们的诊断准确性。最近有几项试点研究调查了将不同类型的患者数据整合到基于 CNN 的皮肤癌分类器中的效果。
本系统评价侧重于当前研究，旨在探讨将图像特征和患者数据信息融合到基于 CNN 的皮肤癌图像分类中对分类器性能的影响。本研究旨在通过评估所使用的患者数据类型、非图像数据的编码方式以及与图像特征的融合方式，以及整合对分类器性能的影响，来探索该领域的研究潜力。
通过 Google Scholar、PubMed、MEDLINE 和 ScienceDirect 筛选出发表在英文期刊上的、涉及基于 CNN 的皮肤癌分类中患者数据整合的同行评审研究。检索词包括皮肤癌分类、卷积神经网络、深度学习、病变、黑色素瘤、元数据、临床信息和患者数据。
共有 11 篇文献符合纳入标准。所有研究均报告了在整合患者数据后，不同皮肤病变分类任务的整体改善。最常用的患者数据是年龄、性别和病变位置。患者数据大多采用独热编码。在将编码后的患者数据与图像特征融合为组合分类器之前和之后，使用深度学习方法对其进行处理的复杂性存在差异。
本研究表明，将患者数据整合到基于 CNN 的诊断算法中具有潜在的益处。然而，患者数据如何具体增强分类性能，特别是在多类分类问题的情况下，仍然不清楚。此外，皮肤科医生使用的大量患者数据仍有待在基于 CNN 的皮肤癌分类的背景下进行分析。在这个很有前途的领域进行进一步的探索性分析可能会优化基于 CNN 的皮肤癌诊断中患者数据的整合，从而使患者受益。
