Department of Automatic Control and Systems Engineering, University Politehnica of Bucharest, Bucharest, Romania.
SkinVision BV, Amsterdam, The Netherlands.
J Eur Acad Dermatol Venereol. 2020 Mar;34(3):648-655. doi: 10.1111/jdv.15935. Epub 2019 Oct 8.
Machine learning algorithms achieve expert-level accuracy in skin lesion classification based on clinical images. However, it is not yet shown whether these algorithms could have high accuracy when embedded in a smartphone app, where image quality is lower and there is high variability in image taking scenarios by users. In the past, these applications were criticized due to lack of accuracy.
In this study, we evaluate the accuracy of the newest version of a smartphone application (SA) for risk assessment of skin lesions.
This SA uses a machine learning algorithm to compute a risk rating. The algorithm is trained on 131 873 images taken by 31 449 users in multiple countries between January 2016 and August 2018 and rated for risk by dermatologists. To evaluate the sensitivity of the algorithm, we use 285 histopathologically validated skin cancer cases (including 138 malignant melanomas), from two previously published clinical studies (195 cases) and from the SA user database (90 cases). We calculate the specificity on a separate set from the SA user database containing 6000 clinically validated benign cases.
The algorithm scored a 95.1% (95% CI, 91.9-97.3%) sensitivity in detecting (pre)malignant conditions (93% for malignant melanoma and 97% for keratinocyte carcinomas and precursors). This level of sensitivity was achieved with a 78.3% (95% CI, 77.2-79.3%) specificity.
This SA provides a high sensitivity to detect skin cancer; however, there is still room for improvement in terms of specificity. Future studies are needed to assess the impact of this SA on the health systems and its users.
机器学习算法在基于临床图像的皮肤病变分类方面达到了专家级别的准确性。然而，目前尚不清楚这些算法在图像质量较低且用户拍摄场景存在高度变异性的智能手机应用程序中是否能够具有高精度。过去，这些应用程序因准确性不足而受到批评。
本研究评估了一款最新版智能手机应用程序（SA）用于皮肤病变风险评估的准确性。
该 SA 使用机器学习算法来计算风险评分。该算法在 2016 年 1 月至 2018 年 8 月期间，通过来自多个国家的 31449 名用户拍摄的 131873 张图像进行训练，并由皮肤科医生对其风险进行评分。为了评估算法的敏感性，我们使用了 285 例经组织病理学验证的皮肤癌病例（包括 138 例恶性黑色素瘤），这些病例分别来自之前发表的两项临床研究（195 例）和 SA 用户数据库（90 例）。我们在 SA 用户数据库中计算了一个单独的特异性数据集，该数据集包含 6000 例经临床验证的良性病例。
该算法在检测（前）恶性病变方面的敏感性为 95.1%（95%CI，91.9-97.3%）（恶性黑色素瘤的敏感性为 93%，角化细胞癌和前体的敏感性为 97%）。这种敏感性水平的特异性为 78.3%（95%CI，77.2-79.3%）。
该 SA 提供了较高的皮肤癌检测敏感性；然而，在特异性方面仍有改进的空间。未来的研究需要评估该 SA 对卫生系统及其用户的影响。
