Yamanaka Syunsuke, Morikawa Koji, Azuma Hiroyuki, Yamanaka Maki, Shimada Yoshimitsu, Wada Toru, Matano Hideyuki, Yamada Naoki, Yamamura Osamu, Hayashi Hiroyuki
Department of Emergency Medicine and General Internal Medicine, University of Fukui Hospital, Fukui, Japan.
Connect Inc., Tokyo, Japan.
Front Med (Lausanne). 2022 Feb 23;9:846525. doi: 10.3389/fmed.2022.846525. eCollection 2022.
Early prediction of oxygen therapy in patients with coronavirus disease 2019 (COVID-19) is vital for triage. Several machine-learning prognostic models for COVID-19 are currently available. However, external validation of these models has rarely been performed. Therefore, most reported predictive performance is optimistic and has a high risk of bias. This study aimed to develop and validate a model that predicts oxygen therapy needs in the early stages of COVID-19 using a sizable multicenter dataset.
This multicenter retrospective study included consecutive COVID-19 hospitalized patients confirmed by a reverse transcription chain reaction in 11 medical institutions in Fukui, Japan. We developed and validated seven machine-learning models (e.g., penalized logistic regression model) using routinely collected data (e.g., demographics, simple blood test). The primary outcome was the need for oxygen therapy (≥1 L/min or SpO ≤ 94%) during hospitalization. C-statistics, calibration slope, and association measures (e.g., sensitivity) evaluated the performance of the model using the test set (randomly selected 20% of data for internal validation). Among these seven models, the machine-learning model that showed the best performance was re-evaluated using an external dataset. We compared the model performances using the A-DROP criteria (modified version of CURB-65) as a conventional method.
Of the 396 patients with COVID-19 for the model development, 102 patients (26%) required oxygen therapy during hospitalization. For internal validation, machine-learning models, except for the -point nearest neighbor, had a higher discrimination ability than the A-DORP criteria ( < 0.01). The XGboost had the highest c-statistic in the internal validation (0.92 vs. 0.69 in A-DROP criteria; < 0.001). For the external validation with 728 temporal independent datasets (106 patients [15%] required oxygen therapy), the XG boost model had a higher c-statistic (0.88 vs. 0.69 in A-DROP criteria; < 0.001).
Machine-learning models demonstrated a more significant performance in predicting the need for oxygen therapy in the early stages of COVID-19.
2019年冠状病毒病（COVID-19）患者氧疗的早期预测对于分诊至关重要。目前有几种针对COVID-19的机器学习预后模型。然而，这些模型很少进行外部验证。因此，大多数报告的预测性能较为乐观，且存在较高的偏差风险。本研究旨在使用一个规模较大的多中心数据集开发并验证一个能够预测COVID-19早期氧疗需求的模型。
这项多中心回顾性研究纳入了日本福井县11家医疗机构中经逆转录链反应确诊的连续COVID-19住院患者。我们使用常规收集的数据（如人口统计学数据、简单血液检测结果）开发并验证了7种机器学习模型（如惩罚逻辑回归模型）。主要结局是住院期间是否需要氧疗（≥1升/分钟或血氧饱和度≤94%）。C统计量、校准斜率和关联度量（如灵敏度）使用测试集（随机选择20%的数据进行内部验证）评估模型性能。在这7种模型中，对表现最佳的机器学习模型使用外部数据集进行重新评估。我们将使用A-DROP标准（CURB-65的修改版）作为传统方法来比较模型性能。
在用于模型开发的396例COVID-19患者中，102例（26%）住院期间需要氧疗。对于内部验证，除了1点最近邻模型外，机器学习模型的辨别能力高于A-DORP标准（P<0.01）。XGBoost在内部验证中具有最高的C统计量（0.92，而A-DROP标准为0.69；P<0.001）。对于使用728个时间独立数据集的外部验证（106例患者[15%]需要氧疗），XGBoost模型具有更高的C统计量（0.88，而A-DROP标准为0.69；P<0.001）。
机器学习模型在预测COVID-19早期氧疗需求方面表现出更显著的性能。
