Division of Internal Medicine, University of Colorado School of Medicine, Aurora, CO, USA.
Colorado Center for Personalized Medicine, University of Colorado School of Medicine, Aurora, CO, USA.
BMC Med Inform Decis Mak. 2020 Oct 2;20(1):252. doi: 10.1186/s12911-020-01268-x.
With cardiovascular disease increasing, substantial research has focused on the development of prediction tools. We compare deep learning and machine learning models to a baseline logistic regression using only 'known' risk factors in predicting incident myocardial infarction (MI) from harmonized EHR data.
Large-scale case-control study with outcome of 6-month incident MI, conducted using the top 800, from an initial 52 k procedures, diagnoses, and medications within the UCHealth system, harmonized to the Observational Medical Outcomes Partnership common data model, performed on 2.27 million patients. We compared several over- and under- sampling techniques to address the imbalance in the dataset. We compared regularized logistics regression, random forest, boosted gradient machines, and shallow and deep neural networks. A baseline model for comparison was a logistic regression using a limited set of 'known' risk factors for MI. Hyper-parameters were identified using 10-fold cross-validation.
Twenty thousand Five hundred and ninety-one patients were diagnosed with MI compared with 2.25 million who did not. A deep neural network with random undersampling provided superior classification compared with other methods. However, the benefit of the deep neural network was only moderate, showing an F1 Score of 0.092 and AUC of 0.835, compared to a logistic regression model using only 'known' risk factors. Calibration for all models was poor despite adequate discrimination, due to overfitting from low frequency of the event of interest.
Our study suggests that DNN may not offer substantial benefit when trained on harmonized data, compared to traditional methods using established risk factors for MI.
随着心血管疾病的增加，大量研究集中在预测工具的开发上。我们将深度学习和机器学习模型与仅使用“已知”风险因素的逻辑回归基线模型进行比较，以从协调的电子健康记录数据中预测心肌梗死（MI）的发生。
这是一项大规模病例对照研究，以 6 个月内发生的 MI 为结局，使用 UCHealth 系统中最初的 52000 个程序、诊断和药物中的前 800 个进行，协调到观察性医疗结果伙伴关系通用数据模型，在 227 万患者上进行。我们比较了几种过采样和欠采样技术，以解决数据集的不平衡问题。我们比较了正则化逻辑回归、随机森林、提升梯度机以及浅层和深层神经网络。比较的基线模型是一个使用有限的 MI 已知风险因素的逻辑回归。使用 10 倍交叉验证确定超参数。
与 2250 万未发生 MI 的患者相比，有 2591 例患者被诊断为 MI。与其他方法相比，使用随机欠采样的深度神经网络提供了更好的分类效果。然而，深度神经网络的优势只是中等的，其 F1 评分仅为 0.092，AUC 为 0.835，而使用仅“已知”风险因素的逻辑回归模型。尽管具有足够的判别能力，但由于感兴趣事件的频率较低，所有模型的校准都很差，这是由过拟合引起的。
我们的研究表明，与使用 MI 既定风险因素的传统方法相比，在对协调数据进行训练时，DNN 可能不会带来实质性的益处。
