Parekh Pranav, Oyeleke Richard, Vishwanath Tejas
Stevens Institute of Technology, Hoboken, NJ, United States.
K.E.M. Hospital, Mumbai, India.
JMIR Dermatol. 2024 Dec 18;7:e59839. doi: 10.2196/59839.
Thus far, considerable research has been focused on classifying a lesion as benign or malignant. However, there is a requirement for quick depth estimation of a lesion for the accurate clinical staging of the lesion. The lesion could be malignant and quickly grow beneath the skin. While biopsy slides provide clear information on lesion depth, it is an emerging domain to find quick and noninvasive methods to estimate depth, particularly based on 2D images.
This study proposes a novel methodology for the depth estimation and visualization of skin lesions. Current diagnostic methods are approximate in determining how much a lesion may have proliferated within the skin. Using color gradients and depth maps, this method will give us a definite estimate and visualization procedure for lesions and other skin issues. We aim to generate 3D holograms of the lesion depth such that dermatologists can better diagnose melanoma.
We started by performing classification using a convolutional neural network (CNN), followed by using explainable artificial intelligence to localize the image features responsible for the CNN output. We used the gradient class activation map approach to perform localization of the lesion from the rest of the image. We applied computer graphics for depth estimation and developing the 3D structure of the lesion. We used the depth from defocus method for depth estimation from single images and Gabor filters for volumetric representation of the depth map. Our novel method, called red spot analysis, measures the degree of infection based on how a conical hologram is constructed. We collaborated with a dermatologist to analyze the 3D hologram output and received feedback on how this method can be introduced to clinical implementation.
The neural model plus the explainable artificial intelligence algorithm achieved an accuracy of 86% in classifying the lesions correctly as benign or malignant. For the entire pipeline, we mapped the benign and malignant cases to their conical representations. We received exceedingly positive feedback while pitching this idea at the King Edward Memorial Institute in India. Dermatologists considered this a potentially useful tool in the depth estimation of lesions. We received a number of ideas for evaluating the technique before it can be introduced to the clinical scene.
When we map the CNN outputs (benign or malignant) to the corresponding hologram, we observe that a malignant lesion has a higher concentration of red spots (infection) in the upper and deeper portions of the skin, and that the malignant cases have deeper conical sections when compared with the benign cases. This proves that the qualitative results map with the initial classification performed by the neural model. The positive feedback provided by the dermatologist suggests that the qualitative conclusion of the method is sufficient.
到目前为止，大量研究集中于将病变分类为良性或恶性。然而，对于准确进行病变临床分期而言，需要快速估计病变的深度。病变可能是恶性的且会在皮肤下迅速生长。虽然活检切片能提供有关病变深度的清晰信息，但寻找快速且非侵入性的深度估计方法，尤其是基于二维图像的方法，仍是一个新兴领域。
本研究提出一种用于皮肤病变深度估计和可视化的新方法。当前的诊断方法在确定病变在皮肤内的增殖程度方面只是近似的。利用颜色梯度和深度图，该方法将为我们提供一种针对病变及其他皮肤问题的明确估计和可视化程序。我们旨在生成病变深度的三维全息图，以便皮肤科医生能更好地诊断黑色素瘤。
我们首先使用卷积神经网络（CNN）进行分类，然后使用可解释人工智能来定位导致CNN输出的图像特征。我们采用梯度类激活映射方法从图像的其余部分中定位病变。我们应用计算机图形学进行深度估计并构建病变的三维结构。我们使用离焦深度方法从单幅图像进行深度估计，并使用Gabor滤波器进行深度图的体积表示。我们的新方法，称为红点分析，基于锥形全息图的构建方式来测量感染程度。我们与一位皮肤科医生合作分析三维全息图输出，并获得了关于如何将该方法引入临床应用的反馈。
神经模型加上可解释人工智能算法在正确将病变分类为良性或恶性方面达到了86%的准确率。对于整个流程，我们将良性和恶性病例映射到它们的锥形表示。当在印度的爱德华国王纪念研究所介绍这个想法时，我们收到了非常积极的反馈。皮肤科医生认为这是病变深度估计中一个潜在有用的工具。在将该技术引入临床场景之前，我们收到了许多评估该技术的想法。
当我们将CNN输出（良性或恶性）映射到相应的全息图时，我们观察到恶性病变在皮肤的上部和深部有更高浓度的红点（感染），并且与良性病例相比，恶性病例的锥形截面更深。这证明了定性结果与神经模型进行的初始分类相匹配。皮肤科医生提供的积极反馈表明该方法的定性结论是充分的。
