Oh Mi-Young, Kim Hee-Soo, Jung Young Mi, Lee Hyung-Chul, Lee Seung-Bo, Lee Seung Mi
Department of Neurology, Sejong General Hospital, Sejong General Hospital, Bucheon-si, Republic of Korea.
Department of Medical Informatics, School of Medicine, Keimyung University, Daegu, Republic of Korea.
J Med Internet Res. 2025 Mar 19;27:e58021. doi: 10.2196/58021.
Machine learning (ML) has the potential to enhance performance by capturing nonlinear interactions. However, ML-based models have some limitations in terms of interpretability.
This study aimed to develop and validate a more comprehensible and efficient ML-based scoring system using SHapley Additive exPlanations (SHAP) values.
We developed and validated the Explainable Automated nonlinear Computation scoring system for Health (EACH) framework score. We developed a CatBoost-based prediction model, identified key features, and automatically detected the top 5 steepest slope change points based on SHAP plots. Subsequently, we developed a scoring system (EACH) and normalized the score. Finally, the EACH score was used to predict perioperative stroke. We developed the EACH score using data from the Seoul National University Hospital cohort and validated it using data from the Boramae Medical Center, which was geographically and temporally different from the development set.
When applied for perioperative stroke prediction among 38,737 patients undergoing noncardiac surgery, the EACH score achieved an area under the curve (AUC) of 0.829 (95% CI 0.753-0.892). In the external validation, the EACH score demonstrated superior predictive performance with an AUC of 0.784 (95% CI 0.694-0.871) compared with a traditional score (AUC=0.528, 95% CI 0.457-0.619) and another ML-based scoring generator (AUC=0.564, 95% CI 0.516-0.612).
The EACH score is a more precise, explainable ML-based risk tool, proven effective in real-world data. The EACH score outperformed traditional scoring system and other prediction models based on different ML techniques in predicting perioperative stroke.
机器学习（ML）有潜力通过捕捉非线性相互作用来提高性能。然而，基于ML的模型在可解释性方面存在一些局限性。
本研究旨在使用SHapley加性解释（SHAP）值开发并验证一个更易理解且高效的基于ML的评分系统。
我们开发并验证了用于健康的可解释自动非线性计算评分系统（EACH）框架分数。我们开发了一个基于CatBoost的预测模型，识别关键特征，并基于SHAP图自动检测出最陡斜率变化点的前5个点。随后，我们开发了一个评分系统（EACH）并对分数进行了标准化。最后，使用EACH分数来预测围手术期卒中。我们使用首尔国立大学医院队列的数据开发了EACH分数，并使用与开发集在地理和时间上不同的博拉梅医疗中心的数据对其进行验证。
在38737例接受非心脏手术的患者中应用于围手术期卒中预测时，EACH分数的曲线下面积（AUC）为0.829（95%CI 0.753 - 0.892）。在外部验证中，与传统分数（AUC = 0.528，95%CI 0.457 - 0.619）和另一个基于ML的评分生成器（AUC = 0.564，95%CI 0.516 - 0.612）相比，EACH分数表现出更好的预测性能，AUC为0.784（95%CI 0.694 - 0.871）。
EACH分数是一种更精确、可解释的基于ML的风险工具，在实际数据中被证明是有效的。在预测围手术期卒中方面，EACH分数优于传统评分系统和基于不同ML技术的其他预测模型。
