Xiaoling Wang, Shengmei Zhu, BingQian Wang, Wen Li, Shuyan Gu, Hanbei Chen, Chenjie Qin, Yao Dai, Jutang Li
Department of Endocrinology, Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai 200092, China.
Department of Pediatric Surgery, Xin Hua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai 200092, China.
Heliyon. 2024 Sep 19;10(19):e37635. doi: 10.1016/j.heliyon.2024.e37635. eCollection 2024 Oct 15.
diabetices foot ulcer (DFU) are serious complications. It is crucial to detect and diagnose DFU early in order to provide timely treatment, improve patient quality of life, and avoid the social and economic consequences. Machine learning techniques can help identify risk factors associated with DFU development.
The aim of this study was to establish correlations between clinical and biochemical risk factors of DFU through local foot examinations based on the construction of predictive models using automated machine learning techniques.
The input dataset consisted of 566 diabetes cases and 50 DFU risk factors, including 9 local foot examinations. 340 patients with Class 0 labeling (low-risk DFU), 226 patients with Class 1 labeling (high-risk DFU). To divide the training group (consisting of 453 cases) and the validation group (consisting of 113 cases), as well as preprocess the data and develop a prediction model, a Monte Carlo cross-validation approach was employed. Furthermore, potential high-risk factors were analyzed using various algorithms, including Bayesian BYS, Multi-Gaussian Weighted Classifier (MGWC), Support Vector Machine (SVM), and Random Forest Classifier (RF). A three-layer machine learning training was constructed, and model performance was estimated using a Confusion Matrix. The top 30 ranking feature variables were ultimately determined. To reinforce the robustness and generalizability of the predictive model, an independent dataset comprising 248 cases was employed for external validation. This validation process evaluated the model's applicability and reliability across diverse populations and clinical settings. Importantly, the external dataset required no additional tuning or adjustment of parameters, enabling an unbiased assessment of the model's generalizability and its capacity to predict the risk of DFU.
The ensemble learning method outperformed individual classifiers in various performance evaluation metrics. Based on the ROC analysis, the AUC of the AutoML model for assessing diabetic foot risk was 88.48 % (74.44-97.83 %). Other results were found to be as follows: 87.23 % (63.33 %-100.00 %) for sensitivity, 87.43 % (70.00 %-100.00 %) for specificity, 87.33 % (76.66 %-95.00 %) for accuracy, 87.69 % (75.00 %-100.00 %) for positive predictive value, and 87.70 % (71.79 %-100.00 %) for negative predictive value. In addition to traditional DFU risk factors such as cardiovascular disorders, peripheral artery disease, and neurological damage, we identified new risk factors such as lower limb varicose veins, history of cerebral infarction, blood urea nitrogen, GFR (Glomerular Filtration Rate), and type of diabetes that may be related to the development of DFU. In the external validation set of 158 samples, originating from an initial 248 with exclusions due to missing labels or features, the model still exhibited strong predictive accuracy. The AUC score of 0.762 indicated a strong discriminatory capability of the model. Furthermore, the Sensitivity and Specificity values provided insights into the model's ability to correctly identify both DFU cases and non-cases, respectively.
The predictive model, developed through AutoML and grounded in local foot examinations, has proven to be a robust and practical instrument for the screening, prediction, and diagnosis of DFU risk. This model not only aids medical practitioners in the identification of potential DFU cases but also plays a pivotal role in mitigating the progression towards adverse outcomes. And the recent successful external validation of our DFU risk prediction model marks a crucial advancement, indicating its readiness for clinical application. This validation reinforces the model's efficacy as an accessible and reliable tool for early DFU risk assessment, thereby facilitating prompt intervention strategies and enhancing overall patient outcomes.
糖尿病足溃疡(DFU)是严重的并发症。早期检测和诊断DFU对于及时治疗、提高患者生活质量以及避免社会和经济后果至关重要。机器学习技术有助于识别与DFU发生相关的风险因素。
本研究的目的是通过基于自动机器学习技术构建预测模型的局部足部检查,建立DFU临床和生化风险因素之间的相关性。
输入数据集包括566例糖尿病病例和50个DFU风险因素,其中包括9项局部足部检查。340例患者标记为0类(低风险DFU),226例患者标记为1类(高风险DFU)。为了划分训练组(由453例病例组成)和验证组(由113例病例组成),以及对数据进行预处理并开发预测模型,采用了蒙特卡洛交叉验证方法。此外,使用各种算法分析潜在的高风险因素,包括贝叶斯BYS、多高斯加权分类器(MGWC)、支持向量机(SVM)和随机森林分类器(RF)。构建了一个三层机器学习训练,并使用混淆矩阵评估模型性能。最终确定了排名前30的特征变量。为了加强预测模型的稳健性和通用性,使用包含248例病例的独立数据集进行外部验证。该验证过程评估了模型在不同人群和临床环境中的适用性和可靠性。重要的是,外部数据集不需要额外的参数调整,能够对模型的通用性及其预测DFU风险的能力进行无偏评估。
在各种性能评估指标中,集成学习方法优于单个分类器。基于ROC分析,用于评估糖尿病足风险的自动机器学习模型的AUC为88.48%(74.44 - 97.83%)。其他结果如下:敏感性为87.23%(63.33% - 100.00%),特异性为87.43%(70.00% - 100.00%),准确性为87.33%(76.66% - 95.00%),阳性预测值为87.69%(75.00% - 100.00%),阴性预测值为87.70%(71.79% - 100.00%)。除了心血管疾病、外周动脉疾病和神经损伤等传统DFU风险因素外,我们还确定了可能与DFU发生相关的新风险因素,如下肢静脉曲张、脑梗死病史、血尿素氮、肾小球滤过率(GFR)和糖尿病类型。在158个样本的外部验证集中,最初有248个样本,由于标签或特征缺失而排除部分样本,该模型仍表现出很强的预测准确性。AUC评分为0.762,表明该模型具有很强的区分能力。此外,敏感性和特异性值分别提供了对模型正确识别DFU病例和非病例能力的见解。
通过自动机器学习开发并基于局部足部检查的预测模型,已被证明是一种用于DFU风险筛查、预测和诊断的强大而实用的工具。该模型不仅有助于医生识别潜在的DFU病例,而且在减轻向不良结局进展方面也起着关键作用。我们的DFU风险预测模型最近成功的外部验证标志着一个关键进展,表明它已准备好用于临床应用。该验证加强了该模型作为一种可获得且可靠的早期DFU风险评估工具的有效性,从而促进及时的干预策略并改善总体患者结局。