Imaging and Computer Vision group, CSIRO Data61, Australia.
Engineering and Design, CSIRO Data61, Australia.
Comput Methods Programs Biomed. 2023 Apr;232:107451. doi: 10.1016/j.cmpb.2023.107451. Epub 2023 Mar 2.
Advanced artificial intelligence and machine learning have great potential to redefine how skin lesions are detected, mapped, tracked and documented. Here, we propose a 3D whole-body imaging system known as 3DSkin-mapper to enable automated detection, evaluation and mapping of skin lesions.
A modular camera rig arranged in a cylindrical configuration was designed to automatically capture images of the entire skin surface of a subject synchronously from multiple angles. Based on the images, we developed algorithms for 3D model reconstruction, data processing and skin lesion detection and tracking based on deep convolutional neural networks. We also introduced a customised, user-friendly, and adaptable interface that enables individuals to interactively visualise, manipulate, and annotate the images. The interface includes built-in features such as mapping 2D skin lesions onto the corresponding 3D model.
The proposed system is developed for skin lesion screening, the focus of this paper is to introduce the system instead of clinical study. Using synthetic and real images we demonstrate the effectiveness of the proposed system by providing multiple views of a target skin lesion, enabling further 3D geometry analysis and longitudinal tracking. Skin lesions are identified as outliers which deserve more attention from a skin cancer physician. Our detector leverages expert annotated labels to learn representations of skin lesions, while capturing the effects of anatomical variability. It takes only a few seconds to capture the entire skin surface, and about half an hour to process and analyse the images.
Our experiments show that the proposed system allows fast and easy whole body 3D imaging. It can be used by dermatological clinics to conduct skin screening, detect and track skin lesions over time, identify suspicious lesions, and document pigmented lesions. The system can potentially save clinicians time and effort significantly. The 3D imaging and analysis has the potential to change the paradigm of whole body photography with many applications in skin diseases, including inflammatory and pigmentary disorders. With reduced time requirements for recording and documenting high-quality skin information, doctors could spend more time providing better-quality treatment based on more detailed and accurate information.
先进的人工智能和机器学习技术具有重新定义皮肤病变检测、绘制、跟踪和记录方式的巨大潜力。在此，我们提出了一种称为 3DSkin-mapper 的 3D 全身成像系统，用于实现皮肤病变的自动检测、评估和绘制。
设计了一种模块化的相机架，以圆柱形配置排列，可自动从多个角度同步捕获受试者整个皮肤表面的图像。基于这些图像，我们开发了用于 3D 模型重建、数据处理以及基于深度卷积神经网络的皮肤病变检测和跟踪的算法。我们还引入了一个定制的、用户友好的、可适应的界面，使个人能够交互地可视化、操作和注释图像。该界面包括内置功能，如将 2D 皮肤病变映射到相应的 3D 模型上。
该系统是为皮肤病变筛查而开发的，本文的重点是介绍该系统，而不是临床研究。我们使用合成和真实图像演示了该系统的有效性，提供了目标皮肤病变的多个视图，从而可以进行进一步的 3D 几何分析和纵向跟踪。皮肤病变被识别为异常值，需要皮肤科医生给予更多关注。我们的检测器利用专家标注的标签来学习皮肤病变的表示，同时捕捉解剖学变异性的影响。拍摄整个皮肤表面只需几秒钟，处理和分析图像则需要大约半小时。
我们的实验表明，该系统允许快速、轻松地进行全身 3D 成像。皮肤科诊所可以使用该系统进行皮肤筛查，随着时间的推移检测和跟踪皮肤病变，识别可疑病变，并记录色素病变。该系统可以显著节省临床医生的时间和精力。3D 成像和分析有可能改变全身摄影的模式，在皮肤病学中有许多应用，包括炎症性和色素性疾病。由于记录和记录高质量皮肤信息的时间要求降低，医生可以花费更多时间根据更详细、更准确的信息提供更高质量的治疗。
