Kagawa Eisuke, Kato Masaya, Oda Noboru, Kunita Eiji, Nagai Michiaki, Yamane Aya, Matsui Shogo, Yoshitomi Yuki, Shimajiri Hiroto, Hirokawa Tatsuya, Ishida Shunsuke, Kurimoto Genki, Dote Keigo
Department of Cardiology, Hiroshima City Asa Hospital, 1-2-1, Kameyamaminami, Asakita-ku, Hiroshima 7310293, Japan.
Department of Cardiology, Hiroshima City North Medical Center Asa Citizens Hospital, Hiroshima, Japan.
Eur Heart J Imaging Methods Pract. 2024 Jun 25;2(1):qyae064. doi: 10.1093/ehjimp/qyae064. eCollection 2024 Jan.
This study assessed an artificial intelligence (AI) model's performance in predicting elevated brain natriuretic peptide (BNP) levels from chest radiograms and its effect on diagnostic performance among healthcare professionals.
Patients who underwent chest radiography and BNP testing on the same day were included. Data were sourced from two hospitals: one for model development, and the other for external testing. Two final ensemble models were developed to predict elevated BNP levels of ≥ 200 pg/mL and ≥ 100 pg/mL, respectively. Humans were evaluated to predict elevated BNP levels, followed by the same test, referring to the AI model's predictions. A total of 8390 images were collected for model creation, and 1713 images, for tests. The AI model achieved an accuracy of 0.855, precision of 0.873, sensitivity of 0.827, specificity of 0.882, f1 score of 0.850, and receiver-operating-characteristics area-under-curve of 0.929. The accuracy of the testing by 35 participants significantly improved from 0.708 ± 0.049 to 0.829 ± 0.069 ( < 0.001) with the AI assistance (an accuracy of 0.920). Without the AI assistance, the accuracy of the veterans in the medical career was higher than that of early-career professionals (0.728 ± 0.051 vs. 0.692 ± 0.042, = 0.030); however, with the AI assistance, the accuracy of the early-career professionals was rather higher than that of the veterans (0.851 ± 0.074 vs. 0.803 ± 0.054, = 0.033).
The AI model can predict elevated BNP levels from chest radiograms and has the potential to improve human performance. The gap in utilizing new tools represents one of the emerging issues.
本研究评估了一种人工智能(AI)模型根据胸部X光片预测脑钠肽(BNP)水平升高的性能及其对医疗专业人员诊断性能的影响。
纳入同日接受胸部X光检查和BNP检测的患者。数据来自两家医院:一家用于模型开发,另一家用于外部测试。开发了两个最终的集成模型,分别预测BNP水平升高≥200 pg/mL和≥100 pg/mL。对人类预测BNP水平升高的能力进行评估,然后进行相同测试,并参考AI模型的预测结果。共收集8390张图像用于模型创建,1713张图像用于测试。AI模型的准确率为0.855,精确率为0.873,灵敏度为0.827,特异性为0.882,F1分数为0.850,受试者工作特征曲线下面积为0.929。在AI辅助下,35名参与者的测试准确率从0.708±0.049显著提高到0.829±0.069(P<0.001)(准确率为0.920)。在没有AI辅助的情况下,医疗职业生涯中的资深人员的准确率高于早期职业专业人员(0.728±0.051对0.692±0.042,P=0.030);然而,在AI辅助下,早期职业专业人员的准确率反而高于资深人员(0.851±0.074对0.803±0.054,P=0.033)。
AI模型可以根据胸部X光片预测BNP水平升高,并且有可能提高人类的诊断性能。在使用新工具方面的差距是一个新出现的问题之一。