Interdisciplinary Program of Biomedical Engineering, Graduate School, Chonnam National University, Yeosu, Republic of Korea.
Department of Digital Medicine, Brain Korea 21 Project, Asan Medical Center, University of Ulsan College of Medicine, Seoul, Republic of Korea.
Clin Chem. 2024 Oct 3;70(10):1256-1267. doi: 10.1093/clinchem/hvae114.
In clinical laboratories, the precision and sensitivity of autoverification technologies are crucial for ensuring reliable diagnostics. Conventional methods have limited sensitivity and applicability, making error detection challenging and reducing laboratory efficiency. This study introduces a machine learning (ML)-based autoverification technology to enhance tumor marker test error detection.
The effectiveness of various ML models was evaluated by analyzing a large data set of 397 751 for model training and internal validation and 215 339 for external validation. Sample misidentification was simulated by random shuffling error-free test results with a 1% error rate to achieve a real-world approximation. The ML models were developed with Bayesian optimization for tuning. Model validation was performed internally at the primary institution and externally at other institutions, comparing the ML models' performance with conventional delta check methods.
Deep neural networks and extreme gradient boosting achieved an area under the receiver operating characteristic curve of 0.834 to 0.903, outperforming that of conventional methods (0.705 to 0.816). External validation by 3 independent laboratories showed that the balanced accuracy of the ML model ranged from 0.760 to 0.836, outperforming the balanced accuracy of 0.670 to 0.773 of the conventional models.
This study addresses limitations regarding the sensitivity of current delta check methods for detection of sample misidentification errors and provides versatile models that mitigate the operational challenges faced by smaller laboratories. Our findings offer a pathway toward more efficient and reliable clinical laboratory testing.
在临床实验室中，自动验证技术的精度和灵敏度对于确保可靠的诊断至关重要。传统方法的灵敏度和适用性有限，使得错误检测具有挑战性，并降低了实验室效率。本研究引入了一种基于机器学习（ML）的自动验证技术，以增强肿瘤标志物检测错误检测。
通过分析 397751 个用于模型训练和内部验证的大数据集和 215339 个用于外部验证的数据集，评估了各种 ML 模型的有效性。通过以 1%的错误率随机打乱无错误测试结果来模拟样本错误识别，以实现对真实世界的近似。使用贝叶斯优化对 ML 模型进行调优。在主要机构内部和其他机构外部进行模型验证，将 ML 模型的性能与传统的 delta 检查方法进行比较。
深度神经网络和极端梯度提升的接收者操作特征曲线下面积为 0.834 至 0.903，优于传统方法（0.705 至 0.816）。由 3 个独立实验室进行的外部验证表明，ML 模型的平衡准确率范围为 0.760 至 0.836，优于传统模型的平衡准确率 0.670 至 0.773。
本研究解决了当前 delta 检查方法在检测样本错误识别方面灵敏度的局限性，并提供了通用的模型，减轻了较小实验室面临的操作挑战。我们的研究结果为更高效、更可靠的临床实验室测试提供了一条途径。
