JLK, Incorporated, Eonju-ro, Gangnam-gu, Seoul, South Korea.
Department of Internal Medicine, Gil Medical Center, Gachon University College of Medicine, Incheon, South Korea.
PLoS One. 2021 Apr 15;16(4):e0249399. doi: 10.1371/journal.pone.0249399. eCollection 2021.
The chest X-ray (CXR) is the most readily available and common imaging modality for the assessment of pneumonia. However, detecting pneumonia from chest radiography is a challenging task, even for experienced radiologists. An artificial intelligence (AI) model might help to diagnose pneumonia from CXR more quickly and accurately. We aim to develop an AI model for pneumonia from CXR images and to evaluate diagnostic performance with external dataset.
To train the pneumonia model, a total of 157,016 CXR images from the National Institutes of Health (NIH) and the Korean National Tuberculosis Association (KNTA) were used (normal vs. pneumonia = 120,722 vs.36,294). An ensemble model of two neural networks with DenseNet classifies each CXR image into pneumonia or not. To test the accuracy of the models, a separate external dataset of pneumonia CXR images (n = 212) from a tertiary university hospital (Gachon University Gil Medical Center GUGMC, Incheon, South Korea) was used; the diagnosis of pneumonia was based on both the chest CT findings and clinical information, and the performance evaluated using the area under the receiver operating characteristic curve (AUC). Moreover, we tested the change of the AI probability score for pneumonia using the follow-up CXR images (7 days after the diagnosis of pneumonia, n = 100).
When the probability scores of the models that have a threshold of 0.5 for pneumonia, two models (models 1 and 4) having different pre-processing parameters on the histogram equalization distribution showed best AUC performances of 0.973 and 0.960, respectively. As expected, the ensemble model of these two models performed better than each of the classification models with 0.983 AUC. Furthermore, the AI probability score change for pneumonia showed a significant difference between improved cases and aggravated cases (Δ = -0.06 ± 0.14 vs. 0.06 ± 0.09, for 85 improved cases and 15 aggravated cases, respectively, P = 0.001) for CXR taken as a 7-day follow-up.
The ensemble model combined two different classification models for pneumonia that performed at 0.983 AUC for an external test dataset from a completely different data source. Furthermore, AI probability scores showed significant changes between cases of different clinical prognosis, which suggest the possibility of increased efficiency and performance of the CXR reading at the diagnosis and follow-up evaluation for pneumonia.
胸部 X 光（CXR）是评估肺炎最常用的成像方式。然而，即使对于有经验的放射科医生来说，从胸部 X 光片中检测肺炎也是一项具有挑战性的任务。人工智能（AI）模型可能有助于更快、更准确地从 CXR 诊断肺炎。我们旨在开发一种用于 CXR 图像的肺炎 AI 模型，并使用外部数据集评估诊断性能。
为了训练肺炎模型，共使用了来自美国国立卫生研究院（NIH）和韩国国家结核病协会（KNTA）的 157016 张 CXR 图像（正常 vs. 肺炎=120722 比 36294）。两个带有 DenseNet 的神经网络的集成模型将每张 CXR 图像分为肺炎或非肺炎。为了测试模型的准确性，使用了一家三级大学医院（韩国仁川加图大学吉尔医疗中心 GUGMC）的另一个外部肺炎 CXR 图像数据集（n=212）；肺炎的诊断基于胸部 CT 结果和临床信息，并使用接收者操作特征曲线下的面积（AUC）进行评估。此外，我们还测试了使用后续 CXR 图像（肺炎诊断后 7 天，n=100）时 AI 肺炎概率评分的变化。
当概率评分模型的阈值为 0.5 时，两个具有不同直方图均衡分布预处理参数的模型（模型 1 和 4）表现出最佳的 AUC 性能，分别为 0.973 和 0.960。如预期的那样，这两个模型的集成模型表现优于每个分类模型，AUC 为 0.983。此外，肺炎的 AI 概率评分变化在改善病例和加重病例之间有显著差异（Δ=-0.06±0.14 比 0.06±0.09，分别为 85 例改善病例和 15 例加重病例，P=0.001），这些病例是在 CXR 拍摄的 7 天随访时的结果。
该集成模型结合了两种不同的肺炎分类模型，在完全不同数据源的外部测试数据集上的 AUC 为 0.983。此外，AI 概率评分在不同临床预后病例之间有显著变化，这表明 CXR 阅读在肺炎的诊断和随访评估中的效率和性能可能会提高。
