Digital Biomarkers for Oncology Group, National Center for Tumor Diseases (NCT), German Cancer Research Center (DKFZ), Heidelberg, Germany.
Department of Clinical-Translational Sciences, Charité University Medicine and Berlin Institute of Health (BIH), Berlin, Germany.
Eur J Cancer. 2021 Mar;145:81-91. doi: 10.1016/j.ejca.2020.11.020. Epub 2021 Jan 7.
A basic requirement for artificial intelligence (AI)-based image analysis systems, which are to be integrated into clinical practice, is a high robustness. Minor changes in how those images are acquired, for example, during routine skin cancer screening, should not change the diagnosis of such assistance systems.
To quantify to what extent minor image perturbations affect the convolutional neural network (CNN)-mediated skin lesion classification and to evaluate three possible solutions for this problem (additional data augmentation, test-time augmentation, anti-aliasing).
We trained three commonly used CNN architectures to differentiate between dermoscopic melanoma and nevus images. Subsequently, their performance and susceptibility to minor changes ('brittleness') was tested on two distinct test sets with multiple images per lesion. For the first set, image changes, such as rotations or zooms, were generated artificially. The second set contained natural changes that stemmed from multiple photographs taken of the same lesions.
All architectures exhibited brittleness on the artificial and natural test set. The three reviewed methods were able to decrease brittleness to varying degrees while still maintaining performance. The observed improvement was greater for the artificial than for the natural test set, where enhancements were minor.
Minor image changes, relatively inconspicuous for humans, can have an effect on the robustness of CNNs differentiating skin lesions. By the methods tested here, this effect can be reduced, but not fully eliminated. Thus, further research to sustain the performance of AI classifiers is needed to facilitate the translation of such systems into the clinic.
人工智能 (AI) 为基础的图像分析系统要集成到临床实践中，一个基本要求是具有高度的稳健性。这些图像获取方式的微小变化，例如在常规皮肤癌筛查期间，不应对这些辅助系统的诊断产生影响。
量化图像的微小扰动在多大程度上影响卷积神经网络 (CNN) 介导的皮肤病变分类，并评估该问题的三种可能解决方案（额外的数据增强、测试时增强、抗混叠）。
我们训练了三个常用的 CNN 架构来区分皮肤镜下黑素瘤和痣的图像。随后，在两个具有多个病变图像的不同测试集上测试它们的性能和对微小变化（“易碎性”）的敏感性。对于第一个测试集，生成了诸如旋转或缩放等图像变化。第二个测试集包含源于同一病变拍摄的多张照片的自然变化。
所有架构在人为和自然测试集上都表现出易碎性。所审查的三种方法能够在不同程度上降低易碎性，同时保持性能。观察到的改进在人为测试集上比自然测试集更大，而在自然测试集上的改进较小。
对于人类来说相对不明显的微小图像变化会影响区分皮肤病变的 CNN 的稳健性。通过这里测试的方法，可以降低这种影响，但不能完全消除。因此，需要进一步研究来维持 AI 分类器的性能，以促进这些系统向临床的转化。
