Department of Radiology, SMG-SNU Boramae Medical Center, Seoul, Korea.
Seoul National University College of Medicine, Seoul, Korea.
Eur Radiol. 2022 May;32(5):3469-3479. doi: 10.1007/s00330-021-08397-5. Epub 2022 Jan 1.
We aim ed to evaluate a commercial artificial intelligence (AI) solution on a multicenter cohort of chest radiographs and to compare physicians' ability to detect and localize referable thoracic abnormalities with and without AI assistance.
In this retrospective diagnostic cohort study, we investigated 6,006 consecutive patients who underwent both chest radiography and CT. We evaluated a commercially available AI solution intended to facilitate the detection of three chest abnormalities (nodule/masses, consolidation, and pneumothorax) against a reference standard to measure its diagnostic performance. Moreover, twelve physicians, including thoracic radiologists, board-certified radiologists, radiology residents, and pulmonologists, assessed a dataset of 230 randomly sampled chest radiographic images. The images were reviewed twice per physician, with and without AI, with a 4-week washout period. We measured the impact of AI assistance on observer's AUC, sensitivity, specificity, and the area under the alternative free-response ROC (AUAFROC).
In the entire set (n = 6,006), the AI solution showed average sensitivity, specificity, and AUC of 0.885, 0.723, and 0.867, respectively. In the test dataset (n = 230), the average AUC and AUAFROC across observers significantly increased with AI assistance (from 0.861 to 0.886; p = 0.003 and from 0.797 to 0.822; p = 0.003, respectively).
The diagnostic performance of the AI solution was found to be acceptable for the images from respiratory outpatient clinics. The diagnostic performance of physicians marginally improved with the use of AI solutions. Further evaluation of AI assistance for chest radiographs using a prospective design is required to prove the efficacy of AI assistance.
• AI assistance for chest radiographs marginally improved physicians' performance in detecting and localizing referable thoracic abnormalities on chest radiographs. • The detection or localization of referable thoracic abnormalities by pulmonologists and radiology residents improved with the use of AI assistance.
我们旨在评估一个商业人工智能（AI）解决方案在多中心胸部 X 光片队列中的表现，并比较医生在有和没有 AI 辅助的情况下检测和定位可归因于胸部的异常的能力。
在这项回顾性诊断队列研究中，我们调查了 6006 例连续接受胸部 X 光和 CT 检查的患者。我们评估了一种商业上可用的 AI 解决方案，旨在通过参考标准来帮助检测三种胸部异常（结节/肿块、实变和气胸），以衡量其诊断性能。此外，12 名医生，包括胸放射科医生、认证放射科医生、放射科住院医师和肺科医生，评估了 230 张随机抽样的胸部 X 光图像数据集。每位医生在 4 周洗脱期内对图像进行两次审查，一次有 AI 辅助，一次没有 AI 辅助。我们测量了 AI 辅助对观察者 AUC、敏感性、特异性和替代自由响应 ROC（AUAFROC）下面积的影响。
在整个数据集（n=6006）中，AI 解决方案的平均敏感性、特异性和 AUC 分别为 0.885、0.723 和 0.867。在测试数据集（n=230）中，观察者的平均 AUC 和 AUAFROC 随着 AI 辅助的使用显著增加（从 0.861 增加到 0.886；p=0.003 和从 0.797 增加到 0.822；p=0.003）。
AI 解决方案的诊断性能被认为是可接受的，可用于呼吸门诊患者的图像。使用 AI 解决方案，医生的诊断性能略有提高。需要使用前瞻性设计进一步评估 AI 辅助对胸部 X 光片的作用，以证明 AI 辅助的有效性。
AI 辅助对胸部 X 光片的使用，略微提高了医生在胸部 X 光片上检测和定位可归因于胸部的异常的能力。
肺科医生和放射科住院医师使用 AI 辅助，可提高对可归因于胸部的异常的检测或定位。
