Draper Laboratory, Cambridge, MA, 02139, USA.
Electrical and Computer Engineering, Northeastern University, Boston, MA, 02115, USA.
Sci Rep. 2021 Feb 11;11(1):3679. doi: 10.1038/s41598-021-82969-9.
Reflectance confocal microscopy (RCM) is a non-invasive imaging tool that reduces the need for invasive histopathology for skin cancer diagnoses by providing high-resolution mosaics showing the architectural patterns of skin, which are used to identify malignancies in-vivo. RCM mosaics are similar to dermatopathology sections, both requiring extensive training to interpret. However, these modalities differ in orientation, as RCM mosaics are horizontal (parallel to the skin surface) while histopathology sections are vertical, and contrast mechanism, RCM with a single (reflectance) mechanism resulting in grayscale images and histopathology with multi-factor color-stained contrast. Image analysis and machine learning methods can potentially provide a diagnostic aid to clinicians to interpret RCM mosaics, eventually helping to ease the adoption and more efficiently utilizing RCM in routine clinical practice. However standard supervised machine learning may require a prohibitive volume of hand-labeled training data. In this paper, we present a weakly supervised machine learning model to perform semantic segmentation of architectural patterns encountered in RCM mosaics. Unlike more widely used fully supervised segmentation models that require pixel-level annotations, which are very labor-demanding and error-prone to obtain, here we focus on training models using only patch-level labels (e.g. a single field of view within an entire mosaic). We segment RCM mosaics into "benign" and "aspecific (nonspecific)" regions, where aspecific regions represent the loss of regular architecture due to injury and/or inflammation, pre-malignancy, or malignancy. We adopt Efficientnet, a deep neural network (DNN) proven to accurately accomplish classification tasks, to generate class activation maps, and use a Gaussian weighting kernel to stitch smaller images back into larger fields of view. The trained DNN achieved an average area under the curve of 0.969, and Dice coefficient of 0.778 showing the feasibility of spatial localization of aspecific regions in RCM images, and making the diagnostics decision model more interpretable to the clinicians.
反射共焦显微镜（RCM）是一种非侵入性成像工具，通过提供显示皮肤结构模式的高分辨率镶嵌图，减少了对皮肤癌诊断的侵入性组织病理学的需求，这些模式图可用于体内识别恶性肿瘤。RCM 镶嵌图类似于皮肤病理学切片，两者都需要广泛的培训才能进行解释。然而，这两种模式在方向上有所不同，因为 RCM 镶嵌图是水平的（平行于皮肤表面），而组织病理学切片是垂直的，并且对比机制也不同，RCM 只有单一（反射）机制，导致灰度图像，而组织病理学则有多因素彩色对比。图像分析和机器学习方法有可能为临床医生提供诊断辅助，以解释 RCM 镶嵌图，最终有助于促进 RCM 在常规临床实践中的采用，并更有效地利用 RCM。然而，标准的监督机器学习可能需要大量的手动标记训练数据。在本文中，我们提出了一种弱监督机器学习模型，用于执行 RCM 镶嵌图中遇到的结构模式的语义分割。与更广泛使用的完全监督分割模型不同，后者需要像素级注释，这些注释非常耗时且容易出错，我们这里专注于仅使用补丁级标签（例如整个镶嵌图中的单个视野）来训练模型。我们将 RCM 镶嵌图分割为“良性”和“非特异性（非特异性）”区域，其中非特异性区域表示由于损伤和/或炎症、癌前病变或恶性肿瘤导致的正常结构丧失。我们采用了 Efficientnet，这是一种经过验证可准确完成分类任务的深度神经网络（DNN），生成类激活图，并使用高斯加权核将较小的图像拼接回较大的视野。经过训练的 DNN 的平均曲线下面积为 0.969，Dice 系数为 0.778，表明 RCM 图像中非特异性区域的空间定位是可行的，并且使诊断决策模型对临床医生更具可解释性。
