Charité Lab for Artificial Intelligence in Medicine-CLAIM, Charité - Universitätsmedizin Berlin, Berlin, Germany.
Centre for Stroke Research Berlin, Charité - Universitätsmedizin Berlin, Berlin, Germany.
PLoS One. 2020 Apr 6;15(4):e0231166. doi: 10.1371/journal.pone.0231166. eCollection 2020.
State-of-the-art machine learning (ML) artificial intelligence methods are increasingly leveraged in clinical predictive modeling to provide clinical decision support systems to physicians. Modern ML approaches such as artificial neural networks (ANNs) and tree boosting often perform better than more traditional methods like logistic regression. On the other hand, these modern methods yield a limited understanding of the resulting predictions. However, in the medical domain, understanding of applied models is essential, in particular, when informing clinical decision support. Thus, in recent years, interpretability methods for modern ML methods have emerged to potentially allow explainable predictions paired with high performance. To our knowledge, we present in this work the first explainability comparison of two modern ML methods, tree boosting and multilayer perceptrons (MLPs), to traditional logistic regression methods using a stroke outcome prediction paradigm. Here, we used clinical features to predict a dichotomized 90 days post-stroke modified Rankin Scale (mRS) score. For interpretability, we evaluated clinical features' importance with regard to predictions using deep Taylor decomposition for MLP, Shapley values for tree boosting and model coefficients for logistic regression. With regard to performance as measured by Area under the Curve (AUC) values on the test dataset, all models performed comparably: Logistic regression AUCs were 0.83, 0.83, 0.81 for three different regularization schemes; tree boosting AUC was 0.81; MLP AUC was 0.83. Importantly, the interpretability analysis demonstrated consistent results across models by rating age and stroke severity consecutively amongst the most important predictive features. For less important features, some differences were observed between the methods. Our analysis suggests that modern machine learning methods can provide explainability which is compatible with domain knowledge interpretation and traditional method rankings. Future work should focus on replication of these findings in other datasets and further testing of different explainability methods.
最先进的机器学习 (ML) 人工智能方法越来越多地被应用于临床预测建模，为医生提供临床决策支持系统。现代 ML 方法，如人工神经网络 (ANNs) 和树增强，通常比逻辑回归等更传统的方法表现更好。另一方面，这些现代方法对产生的预测结果的理解有限。然而，在医学领域，对应用模型的理解是至关重要的，特别是在提供临床决策支持时。因此，近年来，出现了用于现代 ML 方法的可解释性方法，以潜在地实现高性能与可解释预测的结合。据我们所知，我们在这项工作中首次对两种现代 ML 方法（树增强和多层感知机 (MLPs)）与传统逻辑回归方法进行了可解释性比较，使用了中风结果预测范式。在这里，我们使用临床特征来预测 90 天脑卒中改良 Rankin 量表（mRS）评分的二分结果。为了进行可解释性评估，我们使用深度泰勒分解（用于 MLP）、Shapley 值（用于树增强）和模型系数（用于逻辑回归）来评估临床特征对预测的重要性。在测试数据集上，所有模型的性能表现都相当：三种不同正则化方案的逻辑回归 AUC 值分别为 0.83、0.83 和 0.81；树增强的 AUC 值为 0.81；MLP 的 AUC 值为 0.83。重要的是，可解释性分析通过对年龄和中风严重程度进行连续评分，证明了不同模型之间的一致结果，认为这是最重要的预测特征。对于不太重要的特征，在方法之间观察到一些差异。我们的分析表明，现代机器学习方法可以提供可解释性，这与领域知识解释和传统方法排名兼容。未来的工作应该集中在其他数据集上复制这些发现，并进一步测试不同的可解释性方法。
