From the Department of Radiology, Charité, Berlin, Germany.
Charité - Universitätsmedizin Berlin, corporate member of Freie Universität Berlin and Humboldt-Universität zu Berlin, and Berlin Institute of Health (BIH), Department of Radiology, Berlin, Germany.
Invest Radiol. 2021 Aug 1;56(8):525-534. doi: 10.1097/RLI.0000000000000771.
Validation of deep learning models should separately consider bedside chest radiographs (CXRs) as they are the most challenging to interpret, while at the same time the resulting diagnoses are important for managing critically ill patients. Therefore, we aimed to develop and evaluate deep learning models for the identification of clinically relevant abnormalities in bedside CXRs, using reference standards established by computed tomography (CT) and multiple radiologists.
In this retrospective study, a dataset consisting of 18,361 bedside CXRs of patients treated at a level 1 medical center between January 2009 and March 2019 was used. All included CXRs occurred within 24 hours before or after a chest CT. A deep learning algorithm was developed to identify 8 findings on bedside CXRs (cardiac congestion, pleural effusion, air-space opacification, pneumothorax, central venous catheter, thoracic drain, gastric tube, and tracheal tube/cannula). For the training dataset, 17,275 combined labels were extracted from the CXR and CT reports by a deep learning natural language processing (NLP) tool. In case of a disagreement between CXR and CT, human-in-the-loop annotations were used. The test dataset consisted of 583 images, evaluated by 4 radiologists. Performance was assessed by area under the receiver operating characteristic curve analysis, sensitivity, specificity, and positive predictive value.
Areas under the receiver operating characteristic curve for cardiac congestion, pleural effusion, air-space opacification, pneumothorax, central venous catheter, thoracic drain, gastric tube, and tracheal tube/cannula were 0.90 (95% confidence interval [CI], 0.87-0.93; 3 radiologists on the receiver operating characteristic [ROC] curve), 0.95 (95% CI, 0.93-0.96; 3 radiologists on the ROC curve), 0.85 (95% CI, 0.82-0.89; 1 radiologist on the ROC curve), 0.92 (95% CI, 0.89-0.95; 1 radiologist on the ROC curve), 0.99 (95% CI, 0.98-0.99), 0.99 (95% CI, 0.98-0.99), 0.98 (95% CI, 0.97-0.99), and 0.99 (95% CI, 0.98-1.00), respectively.
A deep learning model used specifically for bedside CXRs showed similar performance to expert radiologists. It could therefore be used to detect clinically relevant findings during after-hours and help emergency and intensive care physicians to focus on patient care.
深度学习模型的验证应分别考虑床边胸部 X 线摄影（CXR），因为它们是最具挑战性的解释，同时产生的诊断对于危重症患者的管理非常重要。因此，我们旨在开发和评估深度学习模型，用于识别床边 CXR 中临床相关的异常，使用 CT 和多名放射科医生建立的参考标准。
在这项回顾性研究中，使用了 2009 年 1 月至 2019 年 3 月期间在一级医疗中心接受治疗的 18361 例患者的床边 CXR 数据集。所有包含的 CXR 均在胸部 CT 前或后 24 小时内发生。开发了一种深度学习算法来识别床边 CXR 上的 8 种发现（心脏充血、胸腔积液、气腔混浊、气胸、中心静脉导管、胸腔引流管、胃管和气管导管/套管）。对于训练数据集，从 CXR 和 CT 报告中通过深度学习自然语言处理（NLP）工具提取了 17275 个联合标签。在 CXR 和 CT 之间存在分歧的情况下，使用人机交互注释。测试数据集由 4 名放射科医生评估的 583 张图像组成。通过接受者操作特征曲线分析、灵敏度、特异性和阳性预测值评估性能。
心脏充血、胸腔积液、气腔混浊、气胸、中心静脉导管、胸腔引流管、胃管和气管导管/套管的接受者操作特征曲线下面积分别为 0.90（95%置信区间[CI]，0.87-0.93；3 名放射科医生在接受者操作特征[ROC]曲线上）、0.95（95%CI，0.93-0.96；3 名放射科医生在 ROC 曲线上）、0.85（95%CI，0.82-0.89；1 名放射科医生在 ROC 曲线上）、0.92（95%CI，0.89-0.95；1 名放射科医生在 ROC 曲线上）、0.99（95%CI，0.98-0.99）、0.99（95%CI，0.98-0.99）、0.98（95%CI，0.97-0.99）和 0.99（95%CI，0.98-1.00）。
专门用于床边 CXR 的深度学习模型表现与专家放射科医生相似。因此，它可用于在非工作时间检测临床相关发现，并帮助急诊和重症监护医生专注于患者护理。
