Kwon Osung, Na Wonjun, Kang Heejun, Jun Tae Joon, Kweon Jihoon, Park Gyung-Min, Cho YongHyun, Hur Cinyoung, Chae Jungwoo, Kang Do-Yoon, Lee Pil Hyung, Ahn Jung-Min, Park Duk-Woo, Kang Soo-Jin, Lee Seung-Whan, Lee Cheol Whan, Park Seong-Wook, Park Seung-Jung, Yang Dong Hyun, Kim Young-Hak
Division of Cardiology Department of Internal Medicine, Eunpyeong St Mary's Hospital, Catholic University of Korea, Seoul, Republic of Korea.
Department of Medical Science, Asan Medical Institute of Convergence Science and Technology, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
JMIR Med Inform. 2022 May 11;10(5):e26801. doi: 10.2196/26801.
Although there is a growing interest in prediction models based on electronic medical records (EMRs) to identify patients at risk of adverse cardiac events following invasive coronary treatment, robust models fully utilizing EMR data are limited.
We aimed to develop and validate machine learning (ML) models by using diverse fields of EMR to predict the risk of 30-day adverse cardiac events after percutaneous intervention or bypass surgery.
EMR data of 5,184,565 records of 16,793 patients at a quaternary hospital between 2006 and 2016 were categorized into static basic (eg, demographics), dynamic time-series (eg, laboratory values), and cardiac-specific data (eg, coronary angiography). The data were randomly split into training, tuning, and testing sets in a ratio of 3:1:1. Each model was evaluated with 5-fold cross-validation and with an external EMR-based cohort at a tertiary hospital. Logistic regression (LR), random forest (RF), gradient boosting machine (GBM), and feedforward neural network (FNN) algorithms were applied. The primary outcome was 30-day mortality following invasive treatment.
GBM showed the best performance with area under the receiver operating characteristic curve (AUROC) of 0.99; RF had a similar AUROC of 0.98. AUROCs of FNN and LR were 0.96 and 0.93, respectively. GBM had the highest area under the precision-recall curve (AUPRC) of 0.80, and the AUPRCs of RF, LR, and FNN were 0.73, 0.68, and 0.63, respectively. All models showed low Brier scores of <0.1 as well as highly fitted calibration plots, indicating a good fit of the ML-based models. On external validation, the GBM model demonstrated maximal performance with an AUROC of 0.90, while FNN had an AUROC of 0.85. The AUROCs of LR and RF were slightly lower at 0.80 and 0.79, respectively. The AUPRCs of GBM, LR, and FNN were similar at 0.47, 0.43, and 0.41, respectively, while that of RF was lower at 0.33. Among the categories in the GBM model, time-series dynamic data demonstrated a high AUROC of >0.95, contributing majorly to the excellent results.
Exploiting the diverse fields of the EMR data set, the ML-based 30-day adverse cardiac event prediction models demonstrated outstanding results, and the applied framework could be generalized for various health care prediction models.
尽管基于电子病历（EMR）的预测模型越来越受到关注，用于识别侵入性冠状动脉治疗后发生不良心脏事件风险的患者，但充分利用EMR数据的强大模型却很有限。
我们旨在通过使用EMR的不同领域开发并验证机器学习（ML）模型，以预测经皮介入治疗或搭桥手术后30天不良心脏事件的风险。
2006年至2016年间一家四级医院16793例患者的5184565条EMR记录数据被分类为静态基础数据（如人口统计学数据）、动态时间序列数据（如实验室值）和心脏特异性数据（如冠状动脉造影）。数据按3:1:1的比例随机分为训练集、调整集和测试集。每个模型通过5折交叉验证和在一家三级医院基于外部EMR的队列进行评估。应用了逻辑回归（LR）、随机森林（RF）、梯度提升机（GBM）和前馈神经网络（FNN）算法。主要结局是侵入性治疗后30天的死亡率。
GBM表现最佳，受试者操作特征曲线下面积（AUROC）为0.99；RF的AUROC与之相似，为0.98。FNN和LR的AUROC分别为0.96和0.93。GBM在精确召回率曲线下面积（AUPRC）最高，为0.80，RF、LR和FNN的AUPRC分别为0.73、0.68和0.63。所有模型的Brier评分均<0.1，且校准图拟合度高，表明基于ML的模型拟合良好。在外部验证中，GBM模型表现最佳，AUROC为0.90，而FNN的AUROC为0.85。LR和RF的AUROC略低，分别为0.80和0.79。GBM、LR和FNN的AUPRC相似，分别为0.47、0.43和0.41，而RF的AUPRC较低，为0.33。在GBM模型的类别中，时间序列动态数据的AUROC>0.95，对优异结果贡献较大。
利用EMR数据集的不同领域，基于ML的30天不良心脏事件预测模型取得了出色结果，且所应用的框架可推广用于各种医疗保健预测模型。
