Hunter Joshua G, Bera Kaustav, Shah Neal, Bukhari Syed Muhammad Awais, Marshall Colin, Caovan Danielle, Rosipko Beverly, Gupta Amit
Case Western Reserve University School of Medicine, Cleveland, OH (J.G.H.).
Department of Radiology, University Hospitals Cleveland Medical Center, Cleveland, OH (K.B., N.S., S.M.A.B., C.M., D.C., B.R., A.G.).
Acad Radiol. 2025 Mar;32(3):1165-1174. doi: 10.1016/j.acra.2024.10.012. Epub 2024 Oct 29.
Artificial intelligence (AI) algorithms in radiology capable of detecting urgent findings have gained significant traction in recent years, but the impact of these algorithms on real-world clinical practice remains unclear with need for scientific investigation. Our study investigates the diagnostic accuracy and impact on radiologist report turnaround times of an FDA-approved AI tool for pneumothorax (PTx) detection on inpatient chest X-rays (CXR) in our institution's radiology practice at a large academic medical center.
This retrospective study included 27,397 frontal, single-view CXRs of adult inpatients collected consecutively between August 2020 and April 2021 following deployment of an AI-based PTx detection and picture archiving and communication system (PACS) alert system. 12,728 CXRs were acquired within the AI-integrated system while 14,669 CXRs were acquired outside of the system. Receiver operator characteristic (ROC) analysis was conducted with final radiology reports as the reference standard to evaluate diagnostic accuracy of the AI algorithm in detection of PTx. Wilcoxon rank sum tests were conducted to evaluate the effect of the AI-integrated alert system on radiologist reporting times.
Area under ROC curve (AUC) for the AI tool was.78 with sensitivity of .60 and specificity of .97. When selecting for moderate/large PTx, AUC, sensitivity and specificity increased to .93, .89 and .96, respectively. Median reporting time in CXRs with radiologist-confirmed PTx was reduced by 46% in those with AI integration as compared to those without AI integration (100 vs. 186 min, p < .001).
Real-world deployment of an AI-integrated system capable of detecting PTx and generating alerts within PACS achieved a strong AUC for clinically actionable PTx (i.e., moderate- or large-sized) while substantially reducing median radiologist reporting times, enabling swifter clinical response to a critical but treatable condition.
近年来，放射学领域中能够检测紧急情况的人工智能（AI）算法获得了显著关注，但这些算法对实际临床实践的影响仍不明确，需要进行科学研究。我们的研究调查了一种经美国食品药品监督管理局（FDA）批准的用于检测气胸（PTx）的AI工具在一家大型学术医疗中心的放射科实践中，对住院患者胸部X光片（CXR）的诊断准确性以及对放射科医生报告周转时间的影响。
这项回顾性研究纳入了2020年8月至2021年4月期间连续收集的27397份成年住院患者的正位单视图CXR。在部署基于AI的PTx检测及图像存档与通信系统（PACS）警报系统后，其中12728份CXR是在集成AI的系统内获取的，而14669份CXR是在系统外获取的。以最终放射学报告作为参考标准进行受试者操作特征（ROC）分析，以评估AI算法检测PTx的诊断准确性。进行Wilcoxon秩和检验以评估集成AI的警报系统对放射科医生报告时间的影响。
AI工具的ROC曲线下面积（AUC）为0.78，灵敏度为0.60，特异性为0.97。当选择中度/大型PTx时，AUC、灵敏度和特异性分别提高到0.93、0.89和0.96。与未集成AI的情况相比，在放射科医生确认存在PTx的CXR中，集成AI的情况下中位报告时间减少了46%（100分钟对186分钟，p < 0.001）。
在实际应用中部署能够检测PTx并在PACS内生成警报的集成AI系统，对于临床上可采取行动的PTx（即中度或大型）实现了较高的AUC，同时大幅缩短了放射科医生的中位报告时间，从而能够对危急但可治疗的情况做出更快的临床反应。
