Pienaar M A, Luwes N, Sempa J B, George E, Brown S C
Paediatric Critical Care, Department of Paediatrics and Child Health, School of Clinical Medicine, Faculty of Health Sciences, University of the Free State, Bloemfontein, South Africa.
Central University of Technology, Bloemfontein, South Africa.
South Afr J Crit Care. 2024 Nov 25;40(3):e1398. doi: 10.7196/SAJCC.2024.v40i3.1398. eCollection 2024.
Machine learning (ML) refers to computational algorithms designed to learn from patterns in data to provide insights or predictions related to that data.
Multiple studies report the development of predictive models for triage or identification of critically ill children. In this study, we validate machine learning models developed in South Africa for the identification of critically ill children presenting to a tertiary hospital.
The validation sample comprised 267 patients. The event rate for the study outcome was 0.12. All models demonstrated good discrimination but weak calibration. Artificial neural network 1 (ANN1) had the highest area under the receiver operating characteristic curve (AUROC) with a value of 0.84. ANN2 had the highest area under the precision-recall curve (AUPRC) with a value of 0.65. Decision curve analysis demonstrated that all models were superior to standard strategies of treating all patients or treating no patients at a proposed threshold probability of 10%. Confidence intervals for model performance overlapped considerably. Post hoc model explanations demonstrated that models were logically coherent with clinical knowledge.
Internal validation of the predictive models correlated with model performance in the development study. The models were able to discriminate between critically ill children and non-critically ill children; however, the superiority of one model over the others could not be demonstrated in this study. Therefore, models such as these still require further refinement and external validation before implementation in clinical practice. Indeed, successful implementation of machine learning in practice within the South African setting will require the development of regulatory and infrastructural frameworks in conjunction with the adoption of alternative approaches to electronic data capture, such as the use of mobile devices.
机器学习（ML）是指旨在从数据模式中学习以提供与该数据相关的见解或预测的计算算法。
多项研究报告了用于分诊或识别危重症儿童的预测模型的开发情况。在本研究中，我们对在南非开发的用于识别到三级医院就诊的危重症儿童的机器学习模型进行验证。
验证样本包括267名患者。研究结局的事件发生率为0.12。所有模型均表现出良好的区分度，但校准能力较弱。人工神经网络1（ANN1）在受试者工作特征曲线下面积（AUROC）最高，值为0.84。人工神经网络2在精确召回率曲线下面积（AUPRC）最高，值为0.65。决策曲线分析表明，在提议的阈值概率为10%时，所有模型均优于治疗所有患者或不治疗任何患者的标准策略。模型性能的置信区间有相当大的重叠。事后模型解释表明，模型在逻辑上与临床知识一致。
预测模型的内部验证与开发研究中的模型性能相关。这些模型能够区分危重症儿童和非危重症儿童；然而，在本研究中无法证明一个模型优于其他模型。因此，此类模型在临床实践中实施之前仍需要进一步完善和外部验证。事实上，要在南非环境中成功将机器学习应用于实践，需要结合采用替代电子数据捕获方法（如使用移动设备）来制定监管和基础设施框架。
