Kim Incheol, Rajaraman Sivaramakrishnan, Antani Sameer
Lister Hill National Center for Biomedical Communications, National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894, USA.
Diagnostics (Basel). 2019 Apr 3;9(2):38. doi: 10.3390/diagnostics9020038.
Deep learning (DL) methods are increasingly being applied for developing reliable computer-aided detection (CADe), diagnosis (CADx), and information retrieval algorithms. However, challenges in interpreting and explaining the learned behavior of the DL models hinders their adoption and use in real-world systems. In this study, we propose a novel method called "Class-selective Relevance Mapping" (CRM) for localizing and visualizing discriminative regions of interest (ROI) within a medical image. Such visualizations offer improved explanation of the convolutional neural network (CNN)-based DL model predictions. We demonstrate CRM effectiveness in classifying medical imaging modalities toward automatically labeling them for visual information retrieval applications. The CRM is based on linear sum of incremental mean squared errors (MSE) calculated at the output layer of the CNN model. It measures both positive and negative contributions of each spatial element in the feature maps produced from the last convolution layer leading to correct classification of an input image. A series of experiments on a "multi-modality" CNN model designed for classifying seven different types of image modalities shows that the proposed method is significantly better in detecting and localizing the discriminative ROIs than other state of the art class-activation methods. Further, to visualize its effectiveness we generate "class-specific" ROI maps by averaging the CRM scores of images in each modality class, and characterize the visual explanation through their different size, shape, and location for our multi-modality CNN model that achieved over 98% performance on a dataset constructed from publicly available images.
深度学习（DL）方法越来越多地被应用于开发可靠的计算机辅助检测（CADe）、诊断（CADx）和信息检索算法。然而，解释和说明DL模型的学习行为所面临的挑战阻碍了它们在实际系统中的采用和使用。在本研究中，我们提出了一种名为“类别选择性相关映射”（CRM）的新方法，用于在医学图像中定位和可视化有区别的感兴趣区域（ROI）。这种可视化能更好地解释基于卷积神经网络（CNN）的DL模型预测。我们展示了CRM在对医学成像模态进行分类以自动为视觉信息检索应用标记它们方面的有效性。CRM基于在CNN模型输出层计算的增量均方误差（MSE）的线性和。它测量最后一个卷积层产生的特征图中每个空间元素对正确分类输入图像的正负贡献。在一个为对七种不同类型的图像模态进行分类而设计的“多模态”CNN模型上进行的一系列实验表明，所提出的方法在检测和定位有区别的ROI方面明显优于其他现有技术的类别激活方法。此外，为了可视化其有效性，我们通过对每个模态类中的图像的CRM分数进行平均来生成“特定类别”的ROI图，并通过它们不同的大小、形状和位置来表征视觉解释，我们的多模态CNN模型在由公开可用图像构建的数据集上实现了超过98%的性能。
