Mahbod Amirreza, Schaefer Gerald, Wang Chunliang, Dorffner Georg, Ecker Rupert, Ellinger Isabella
Institute for Pathophysiology and Allergy Research, Medical University of Vienna, Vienna, Austria; Research and Development Department of TissueGnostics GmbH, Vienna, Austria.
Department of Computer Science, Loughborough University, Loughborough, United Kingdom.
Comput Methods Programs Biomed. 2020 Sep;193:105475. doi: 10.1016/j.cmpb.2020.105475. Epub 2020 Mar 21.
Skin cancer is among the most common cancer types in the white population and consequently computer aided methods for skin lesion classification based on dermoscopic images are of great interest. A promising approach for this uses transfer learning to adapt pre-trained convolutional neural networks (CNNs) for skin lesion diagnosis. Since pre-training commonly occurs with natural images of a fixed image resolution and these training images are usually significantly smaller than dermoscopic images, downsampling or cropping of skin lesion images is required. This however may result in a loss of useful medical information, while the ideal resizing or cropping factor of dermoscopic images for the fine-tuning process remains unknown.
We investigate the effect of image size for skin lesion classification based on pre-trained CNNs and transfer learning. Dermoscopic images from the International Skin Imaging Collaboration (ISIC) skin lesion classification challenge datasets are either resized to or cropped at six different sizes ranging from 224 × 224 to 450 × 450. The resulting classification performance of three well established CNNs, namely EfficientNetB0, EfficientNetB1 and SeReNeXt-50 is explored. We also propose and evaluate a multi-scale multi-CNN (MSM-CNN) fusion approach based on a three-level ensemble strategy that utilises the three network architectures trained on cropped dermoscopic images of various scales.
Our results show that image cropping is a better strategy compared to image resizing delivering superior classification performance at all explored image scales. Moreover, fusing the results of all three fine-tuned networks using cropped images at all six scales in the proposed MSM-CNN approach boosts the classification performance compared to a single network or a single image scale. On the ISIC 2018 skin lesion classification challenge test set, our MSM-CNN algorithm yields a balanced multi-class accuracy of 86.2% making it the currently second ranked algorithm on the live leaderboard.
We confirm that the image size has an effect on skin lesion classification performance when employing transfer learning of CNNs. We also show that image cropping results in better performance compared to image resizing. Finally, a straightforward ensembling approach that fuses the results from images cropped at six scales and three fine-tuned CNNs is shown to lead to the best classification performance.
皮肤癌是白人群体中最常见的癌症类型之一，因此基于皮肤镜图像的皮肤病变分类计算机辅助方法备受关注。一种很有前景的方法是使用迁移学习来使预训练的卷积神经网络（CNN）适用于皮肤病变诊断。由于预训练通常是针对固定图像分辨率的自然图像进行的，且这些训练图像通常比皮肤镜图像小得多，所以需要对皮肤病变图像进行下采样或裁剪。然而，这可能会导致有用医学信息的丢失，同时用于微调过程的皮肤镜图像的理想调整大小或裁剪因子仍然未知。
我们基于预训练的CNN和迁移学习研究图像大小对皮肤病变分类的影响。来自国际皮肤成像协作组织（ISIC）皮肤病变分类挑战数据集的皮肤镜图像被调整大小或裁剪为从224×224到450×450的六种不同尺寸。探索了三种成熟的CNN，即EfficientNetB0、EfficientNetB1和SeReNeXt - 50的分类性能。我们还提出并评估了一种基于三级集成策略的多尺度多CNN（MSM - CNN）融合方法，该策略利用在不同尺度裁剪的皮肤镜图像上训练的三种网络架构。
我们的结果表明，与图像调整大小相比，图像裁剪是一种更好的策略，在所有探索的图像尺度上都能提供更好的分类性能。此外，在所提出的MSM - CNN方法中，使用所有六个尺度裁剪的图像融合所有三个微调网络的结果，与单个网络或单个图像尺度相比，提高了分类性能。在ISIC 2018皮肤病变分类挑战测试集上，我们的MSM - CNN算法产生了86.2%的平衡多类准确率，使其在实时排行榜上目前排名第二。
我们证实，在采用CNN迁移学习时，图像大小会影响皮肤病变分类性能。我们还表明，与图像调整大小相比，图像裁剪能带来更好的性能。最后，一种简单的集成方法，即融合在六个尺度裁剪的图像和三个微调CNN的结果，被证明能带来最佳的分类性能。
