Storås Andrea Marheim, Mæland Steffen, Isaksen Jonas L, Hicks Steven Alexander, Thambawita Vajira, Graff Claus, Hammer Hugo Lewi, Halvorsen Pål, Riegler Michael Alexander, Kanters Jørgen K
Department of Holistic Systems, SimulaMet, 0170 Oslo, Norway.
Department of Computer Science, Oslo Metropolitan University, 0167 Oslo, Norway.
J Am Med Inform Assoc. 2025 Jan 1;32(1):79-88. doi: 10.1093/jamia/ocae280.
Evaluate popular explanation methods using heatmap visualizations to explain the predictions of deep neural networks for electrocardiogram (ECG) analysis and provide recommendations for selection of explanations methods.
A residual deep neural network was trained on ECGs to predict intervals and amplitudes. Nine commonly used explanation methods (Saliency, Deconvolution, Guided backpropagation, Gradient SHAP, SmoothGrad, Input × gradient, DeepLIFT, Integrated gradients, GradCAM) were qualitatively evaluated by medical experts and objectively evaluated using a perturbation-based method.
No single explanation method consistently outperformed the other methods, but some methods were clearly inferior. We found considerable disagreement between the human expert evaluation and the objective evaluation by perturbation.
The best explanation method depended on the ECG measure. To ensure that future explanations of deep neural networks for medical data analyses are useful to medical experts, data scientists developing new explanation methods should collaborate tightly with domain experts. Because there is no explanation method that performs best in all use cases, several methods should be applied.
Several explanation methods should be used to determine the most suitable approach.
使用热图可视化评估常用的解释方法，以解释深度神经网络对心电图（ECG）分析的预测，并为解释方法的选择提供建议。
在心电图上训练一个残差深度神经网络，以预测间期和振幅。医学专家对九种常用的解释方法（显著性、反卷积、引导反向传播、梯度SHAP、平滑梯度、输入×梯度、深度提升、集成梯度、GradCAM）进行了定性评估，并使用基于扰动的方法进行了客观评估。
没有一种解释方法始终优于其他方法，但有些方法明显较差。我们发现人类专家评估与基于扰动的客观评估之间存在相当大的分歧。
最佳解释方法取决于心电图测量指标。为确保未来深度神经网络对医学数据分析的解释对医学专家有用，开发新解释方法的数据科学家应与领域专家紧密合作。由于没有一种解释方法在所有用例中都表现最佳，因此应应用多种方法。
应使用多种解释方法来确定最合适的方法。
