Department of Emergency and Critical Care Medicine, Taipei Medical University Hospital, Taipei, Taiwan.
Department of Medicine Education, Taipei Medical University Hospital, Taipei, Taiwan.
Int J Med Inform. 2020 Sep;141:104176. doi: 10.1016/j.ijmedinf.2020.104176. Epub 2020 May 21.
Severe sepsis and septic shock are still the leading causes of death in Intensive Care Units (ICUs), and timely diagnosis is crucial for treatment outcomes. The progression of electronic medical records (EMR) offers the possibility of storing a large quantity of clinical data that can facilitate the development of artificial intelligence (AI) in medicine. However, several difficulties, such as poor structure and heterogenicity of the raw EMR data, are encountered when introducing AI with ICU data. Labor-intensive work, including manual data entry, personal medical records sorting, and laboratory results interpretation may hinder the progress of AI. In this article, we introduce the developing of an AI algorithm designed for sepsis diagnosis using pre-selected features; and compare the performance of the AI algorithm with SOFA score based diagnostic method.
This is a prospective open-label cohort study. A specialized EMR, named TED_ICU, was implemented for continuous data recording. One hundred six clinical features relevant to sepsis diagnosis were selected prospectively. A labeling work to allocate SEPSIS or NON_SEPSIS status for each ICU patient was performed by the in-charge intensivist according to SEPSIS-3 criteria, along with the automatic recording of selected features every day by TED_ICU. Afterward, we use de-identified data to develop the AI algorithm. Several machine learning methods were evaluated using 5-fold cross-validation, and XGBoost, a decision-tree based algorithm was adopted for our AI algorithm development due to best performance.
The study was conducted between August 2018 and December 2018 for the first stage of analysis. We collected 1588 instances, including 444 SEPSIS and 1144 NON-SEPSIS, from 434 patients. The 434 patients included 259 (59.6%) male patients and 175 female patients. The mean age was 67.6-year-old, and the mean APACHE II score was 13.8. The SEPSIS cohort had a higher SOFA score and increased use of organ support treatment. The AI algorithm was developed with a shuffle method using 80% of the instances for training and 20% for testing. The established AI algorithm achieved the following: accuracy = 82% ± 1%; sensitivity = 65% ± 5%; specificity = 88% ± 2%; precision = 67% ± 3%; and F1 = 0.66 ± 0.02. The area under the receiver operating characteristic curve (AUROC) was approximately 0.89. The SOFA score was used on the same 1588 instances for sepsis diagnosis, and the result was inferior to our AI algorithm (AUROC = 0.596).
Using real-time data, collected by EMR, from the ICU daily practice, our AI algorithm established with pre-selected features and XGBoost can provide a timely diagnosis of sepsis with an accuracy greater than 80%. AI algorithm also outperforms the SOFA score in sepsis diagnosis and exhibits practicality as clinicians can deploy appropriate treatment earlier. The early and precise response of this AI algorithm will result in cost reduction, outcome improvement, and benefit for healthcare systems, medical staff, and patients as well.
严重脓毒症和脓毒性休克仍然是重症监护病房（ICU）死亡的主要原因，及时诊断对于治疗结果至关重要。电子病历（EMR）的发展提供了存储大量临床数据的可能性，这有助于在医学中开发人工智能（AI）。然而，在引入 ICU 数据的 AI 时，会遇到电子病历数据原始结构差和异质性等困难。包括手动数据输入、个人病历分类和实验室结果解释在内的劳动密集型工作可能会阻碍 AI 的进展。在本文中，我们介绍了一种使用预选特征设计的用于脓毒症诊断的 AI 算法的开发；并比较了 AI 算法与 SOFA 评分基于诊断方法的性能。
这是一项前瞻性开放标签队列研究。专门开发了一个名为 TED_ICU 的 EMR，用于连续数据记录。前瞻性选择了 106 个与脓毒症诊断相关的临床特征。根据 SEPSIS-3 标准，由主治重症监护医生对每位 ICU 患者进行 SEPSIS 或 NON_SEPSIS 状态的标记工作，并由 TED_ICU 每天自动记录选定的特征。然后，我们使用去标识数据来开发 AI 算法。使用 5 折交叉验证评估了几种机器学习方法，由于性能最佳，我们采用了基于决策树的 XGBoost 算法来开发我们的 AI 算法。
该研究于 2018 年 8 月至 2018 年 12 月进行了第一阶段分析。我们从 434 名患者中收集了 1588 例患者，包括 444 例 SEPSIS 和 1144 例 NON_SEPSIS。434 名患者中包括 259 名（59.6%）男性患者和 175 名女性患者。平均年龄为 67.6 岁，平均 APACHE II 评分为 13.8。SEPSIS 组的 SOFA 评分更高，并且使用器官支持治疗的比例增加。使用 80%的实例进行训练和 20%的实例进行测试的 shuffle 方法开发了 AI 算法。建立的 AI 算法取得了以下结果：准确率=82%±1%；灵敏度=65%±5%；特异性=88%±2%；精确率=67%±3%；F1=0.66±0.02。接受者操作特征曲线（AUROC）的面积约为 0.89。SOFA 评分用于对相同的 1588 例 SEPSIS 患者进行脓毒症诊断，结果不如我们的 AI 算法（AUROC=0.596）。
使用 EMR 实时收集的 ICU 日常实践数据，我们使用预选特征和 XGBoost 建立的 AI 算法可以提供超过 80%的准确性，及时诊断脓毒症。AI 算法在脓毒症诊断中的性能也优于 SOFA 评分，并且具有实用性，因为临床医生可以更早地部署适当的治疗。这种 AI 算法的早期和精确响应将导致成本降低、结果改善，并使医疗保健系统、医务人员和患者受益。
