Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark; Department of Intensive Care, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark.
Novo Nordisk Foundation Center for Protein Research, Faculty of Health and Medical Sciences, University of Copenhagen, Copenhagen, Denmark.
Lancet Digit Health. 2020 Apr;2(4):e179-e191. doi: 10.1016/S2589-7500(20)30018-2. Epub 2020 Mar 12.
BACKGROUND: Many mortality prediction models have been developed for patients in intensive care units (ICUs); most are based on data available at ICU admission. We investigated whether machine learning methods using analyses of time-series data improved mortality prognostication for patients in the ICU by providing real-time predictions of 90-day mortality. In addition, we examined to what extent such a dynamic model could be made interpretable by quantifying and visualising the features that drive the predictions at different timepoints. METHODS: Based on the Simplified Acute Physiology Score (SAPS) III variables, we trained a machine learning model on longitudinal data from patients admitted to four ICUs in the Capital Region, Denmark, between 2011 and 2016. We included all patients older than 16 years of age, with an ICU stay lasting more than 1 h, and who had a Danish civil registration number to enable 90-day follow-up. We leveraged static data and physiological time-series data from electronic health records and the Danish National Patient Registry. A recurrent neural network was trained with a temporal resolution of 1 h. The model was internally validated using the holdout method with 20% of the training dataset and externally validated using previously unseen data from a fifth hospital in Denmark. Its performance was assessed with the Matthews correlation coefficient (MCC) and area under the receiver operating characteristic curve (AUROC) as metrics, using bootstrapping with 1000 samples with replacement to construct 95% CIs. A Shapley additive explanations algorithm was applied to the prediction model to obtain explanations of the features that drive patient-specific predictions, and the contributions of each of the 44 features in the model were analysed and compared with the variables in the original SAPS III model. FINDINGS: From a dataset containing 15 615 ICU admissions of 12 616 patients, we included 14 190 admissions of 11 492 patients in our analysis. Overall, 90-day mortality was 33·1% (3802 patients). The deep learning model showed a predictive performance on the holdout testing dataset that improved over the timecourse of an ICU stay: MCC 0·29 (95% CI 0·25-0·33) and AUROC 0·73 (0·71-0·74) at admission, 0·43 (0·40-0·47) and 0·82 (0·80-0·84) after 24 h, 0·50 (0·46-0·53) and 0·85 (0·84-0·87) after 72 h, and 0·57 (0·54-0·60) and 0·88 (0·87-0·89) at the time of discharge. The model exhibited good calibration properties. These results were validated in an external validation cohort of 5827 patients with 6748 admissions: MCC 0·29 (95% CI 0·27-0·32) and AUROC 0·75 (0·73-0·76) at admission, 0·41 (0·39-0·44) and 0·80 (0·79-0·81) after 24 h, 0·46 (0·43-0·48) and 0·82 (0·81-0·83) after 72 h, and 0·47 (0·44-0·49) and 0·83 (0·82-0·84) at the time of discharge. INTERPRETATION: The prediction of 90-day mortality improved with 1-h sampling intervals during the ICU stay. The dynamic risk prediction can also be explained for an individual patient, visualising the features contributing to the prediction at any point in time. This explanation allows the clinician to determine whether there are elements in the current patient state and care that are potentially actionable, thus making the model suitable for further validation as a clinical tool. FUNDING: Novo Nordisk Foundation and the Innovation Fund Denmark.
背景:许多用于重症监护病房(ICU)患者的死亡率预测模型已经开发出来;大多数都是基于 ICU 入院时可获得的数据。我们研究了使用时间序列数据分析的机器学习方法是否可以通过实时预测 90 天死亡率来改善 ICU 患者的死亡率预测。此外,我们还通过量化和可视化不同时间点的预测所驱动的特征,来检查这种动态模型在多大程度上可以具有可解释性。
方法:基于简化急性生理学评分(SAPS)III 变量,我们在丹麦首都地区 2011 年至 2016 年间收治的 4 家 ICU 的纵向数据上训练了一个机器学习模型。我们纳入了所有年龄大于 16 岁、入住 ICU 时间超过 1 小时、且有丹麦公民登记号以进行 90 天随访的患者。我们利用了电子病历和丹麦国家患者登记处的静态数据和生理时间序列数据。一个递归神经网络以 1 小时的时间分辨率进行训练。该模型使用 20%的训练数据集的保留方法进行内部验证,并使用丹麦第五家医院的以前未见的数据进行外部验证。使用 bootstrap 方法(替换 1000 个样本)构建 95%置信区间来评估其性能,使用 Matthews 相关系数(MCC)和接收者操作特征曲线下面积(AUROC)作为指标。应用 Shapley 加性解释算法对预测模型进行分析,以获得驱动患者特定预测的特征的解释,并分析和比较模型中 44 个特征中的每一个与原始 SAPS III 模型中的变量的贡献。
结果:在包含 15615 例 ICU 入住和 12616 例患者的数据集,我们纳入了我们分析中的 14190 例 ICU 入住和 11492 例患者。总体而言,90 天死亡率为 33.1%(3802 例患者)。深度学习模型在 ICU 入住期间的时间进程上显示出了预测性能的提高:在入院时,MCC 为 0.29(95%CI 0.25-0.33)和 AUROC 为 0.73(0.71-0.74),24 小时后为 0.43(0.40-0.47)和 AUROC 为 0.82(0.80-0.84),72 小时后为 0.50(0.46-0.53)和 AUROC 为 0.85(0.84-0.87),出院时为 0.57(0.54-0.60)和 AUROC 为 0.88(0.87-0.89)。该模型表现出良好的校准特性。这些结果在包含 5827 例患者和 6748 例入住的外部验证队列中得到了验证:在入院时,MCC 为 0.29(95%CI 0.27-0.32)和 AUROC 为 0.75(0.73-0.76),24 小时后为 0.41(0.39-0.44)和 AUROC 为 0.80(0.79-0.81),72 小时后为 0.46(0.43-0.48)和 AUROC 为 0.82(0.81-0.83),出院时为 0.47(0.44-0.49)和 AUROC 为 0.83(0.82-0.84)。
解释:随着 ICU 入住期间 1 小时采样间隔的增加,90 天死亡率的预测得到了改善。动态风险预测还可以为个别患者进行解释,可视化不同时间点的预测所驱动的特征。这种解释使临床医生能够确定当前患者状态和护理中是否有潜在可操作的元素,从而使该模型适合进一步验证作为临床工具。
资助:诺和诺德基金会和丹麦创新基金会。
Comput Methods Programs Biomed. 2024-11
Int J Med Inform. 2017-10-5
BMC Med Inform Decis Mak. 2020-10-2
Am J Transl Res. 2025-4-15
BMC Med Inform Decis Mak. 2025-4-15