Yesil Meryem Rumeysa, Talli Ilaria, Pelloso Michela, Cosma Chiara, Pangrazzi Elisa, Plebani Mario, Ustundag Yasemin, Padoan Andrea
Department of Medical Biochemistry, University of Health Sciences, Bursa Yuksek Ihtisas Training and Research Hospital, Bursa, Türkiye.
Department of Medicine (DIMED), University of Padova, Padova, Italy.
Clin Chem Lab Med. 2025 May 28. doi: 10.1515/cclm-2025-0491.
Machine learning (ML) models, using laboratory data, support early sepsis prediction. However, analytical bias in laboratory measurements can compromise their performance and validity in real-world settings. We aimed to evaluate how analytically acceptable bias may affect the validity and generalizability of ML models trained on laboratory data.
A support vector machine model (SVM) for sepsis prediction was developed using complete blood count and erythrocyte sedimentation rate data from outpatients (CS, n=104) and patients from acute inflammatory status wards (SS, n=107). Twenty-six combinations were derived by white blood cells (WBC), platelets (PLT), and erythrocyte sedimentation rate (ESR) biases from analytical performance specifications (APS). The diagnostic performances of the 26 conditions tested were compared to the original dataset.
SVM performance of the original dataset was AUC 90.6 % [95 %CI: 80.6-98.7 %]. Minimum, desirable and optimum acceptable biases for WBC were 7.7 , 5.1 and 2.6 %, respectively, for PLT were 6.7 , 4.5 and 2.2 %, respectively and for ESR were 31.6 , 21.1 and 10.5 %, respectively. Across all conditions, AUC varied from 89.8 % [95 %CI: 79.0-97.7 %] (for PLT bias -6.7 %), to 89.5 % [95 %CI: 79.1-98.0 %] (for ESR Bias +31.6 %) to 90.4 % [95 %CI: 79.3-98.4 %] (for WBC Bias -5.1 %). Using a combination of biases, the lowest AUC was 87.8 % [95 %CI: 75.9-96.6 %]. No statistically significant differences were observed for AUC (p>0.05).
Bias can influence model performance depending on the parameters and their combinations. Developing new validation strategies to assess the impact of analytical bias on laboratory data in ML models could improve their reliability.
利用实验室数据的机器学习（ML）模型有助于早期脓毒症预测。然而，实验室测量中的分析偏差可能会损害其在实际环境中的性能和有效性。我们旨在评估分析上可接受的偏差如何影响基于实验室数据训练的ML模型的有效性和通用性。
利用门诊患者（CS，n = 104）和急性炎症状态病房患者（SS，n = 107）的全血细胞计数和红细胞沉降率数据，开发了一种用于脓毒症预测的支持向量机模型（SVM）。根据分析性能规范（APS）中的白细胞（WBC）、血小板（PLT）和红细胞沉降率（ESR）偏差，得出26种组合。将测试的26种情况的诊断性能与原始数据集进行比较。
原始数据集的SVM性能为AUC 90.6% [95%CI：80.6 - 98.7%]。WBC的最小、理想和最佳可接受偏差分别为7.7%、5.1%和2.6%，PLT分别为6.7%、4.5%和2.2%，ESR分别为31.6%、21.1%和10.5%。在所有情况下，AUC从89.8% [95%CI：79.0 - 97.7%]（PLT偏差 - 6.7%）到89.5% [95%CI：79.1 - 98.0%]（ESR偏差 + 31.6%）再到90.4% [95%CI：79.3 - 98.4%]（WBC偏差 - 5.1%）不等。使用偏差组合时，最低AUC为87.8% [95%CI：75.9 - 96.6%]。未观察到AUC有统计学显著差异（p>0.05）。
偏差可根据参数及其组合影响模型性能。制定新的验证策略以评估分析偏差对ML模型中实验室数据的影响，可提高其可靠性。
