Wang Xin, Yang Yu-Qing, Hong Xin-Yu, Liu Si-Hua, Li Jian-Chu, Chen Ting, Shi Ju-Hong
Department of Ultrasound, Peking Union Medical College Hospital, Beijing, China.
Chinese Academy of Medical Sciences, Peking Union Medical College, Beijing, China.
BMC Med Inform Decis Mak. 2024 Dec 30;24(1):413. doi: 10.1186/s12911-024-02834-3.
Inpatients with high risk of venous thromboembolism (VTE) usually face serious threats to their health and economic conditions. Many studies using machine learning (ML) models to predict VTE risk overlook the impact of class-imbalance problem due to the low incidence rate of VTE, resulting in inferior and unstable model performance, which hinders their ability to replace the Padua model, a widely used linear weighted model in clinic. Our study aims to develop a new VTE risk assessment model suitable for Chinese medical inpatients.
3284 inpatients in the medical department of Peking Union Medical College Hospital (PUMCH) from January 2014 to June 2016 were collected. The training and test set were divided based on the admission time and inpatients from May 2016 to June 2016 were included as the test dataset. We explained the class imbalance problem from a clinical perspective and defined a new term, "fuzzy population", to elaborate and model this phenomenon. By considering the "fuzzy population", a new ML VTE risk assessment model was built through population splitting. Sensitivity and specificity of our method was compared with five ML models (support vector machine (SVM), random forest (RF), gradient boosting decision tree (GBDT), logistic regression (LR), and XGBoost) and the Padua model.
The 'fuzzy population' phenomenon was explained and verified on the VTE dataset. The proposed model achieved higher specificity (64.94% vs. 63.30%) and the same sensitivity (90.24% vs. 90.24%) on test data than the Padua model. Other five ML models couldn't simultaneously surpass the Padua's sensitivity and specificity. Besides, our model was more robust than five ML models and its standard deviations of sensitivities and specificities were smaller. Adjusting the distribution of negative samples in the training set based on the 'fuzzy population' would exacerbate the instability of performance of five ML models, which limited the application of ML methods in clinic.
The proposed model achieved higher sensitivity and specificity than the Padua model, and better robustness than traditional ML models. This study built a population-split-based ML model of VTE by modeling the class-imbalance problem and it can be applied more broadly in risk assessment of other diseases.
静脉血栓栓塞症（VTE）高危住院患者通常面临严重的健康和经济状况威胁。许多使用机器学习（ML）模型预测VTE风险的研究，由于VTE发病率低而忽视了类不平衡问题的影响，导致模型性能较差且不稳定，这阻碍了它们取代临床上广泛使用的线性加权模型Padua模型的能力。我们的研究旨在开发一种适用于中国内科住院患者的新型VTE风险评估模型。
收集了2014年1月至2016年6月在北京协和医院（PUMCH）内科住院的3284例患者。根据入院时间划分训练集和测试集，并将2016年5月至6月的住院患者作为测试数据集。我们从临床角度解释了类不平衡问题，并定义了一个新术语“模糊人群”来阐述和模拟这一现象。通过考虑“模糊人群”，通过人群划分构建了一个新的ML VTE风险评估模型。将我们方法的敏感性和特异性与五个ML模型（支持向量机（SVM）、随机森林（RF）、梯度提升决策树（GBDT）、逻辑回归（LR）和XGBoost）以及Padua模型进行了比较。
在VTE数据集上解释并验证了“模糊人群”现象。在测试数据上，所提出的模型比Padua模型具有更高的特异性（64.94%对63.30%）和相同的敏感性（90.24%对90.24%）。其他五个ML模型不能同时超过Padua模型的敏感性和特异性。此外，我们的模型比五个ML模型更稳健，其敏感性和特异性的标准差更小。基于“模糊人群”调整训练集中阴性样本的分布会加剧五个ML模型性能的不稳定，这限制了ML方法在临床上的应用。
所提出的模型比Padua模型具有更高的敏感性和特异性，并且比传统ML模型具有更好的稳健性。本研究通过对类不平衡问题进行建模，构建了基于人群划分的VTE的ML模型，并且它可以更广泛地应用于其他疾病的风险评估。
