Chang Tien-Yu, Chou Ting Ywan, Jen I-An, Yuh Yeong-Seng
Department of Radiology, Cheng-Hsin General Hospital, Taipei, Taiwan, ROC.
Department of Radiology, Cardinal Tien General Hospital, Taipei, Taiwan, ROC.
J Chin Med Assoc. 2025 Jul 1;88(7):530-537. doi: 10.1097/JCMA.0000000000001248. Epub 2025 May 15.
Artificial intelligence (AI) algorithms can provide rapid and precise radiographic bone age (BA) assessment. This study assessed the effects of an AI algorithm on the BA assessment performance of radiologists, and evaluated how automation bias could affect radiologists.
In this prospective randomized crossover study, six radiologists with varying levels of experience (senior, mid-level, and junior) assessed cases from a test set of 200 standard BA radiographs. The test set was equally divided into two subsets: datasets A and B. Each radiologist assessed BA independently without AI assistance (A- B-) and with AI assistance (A+ B+). We used the mean of assessments made by two experts as the ground truth for accuracy assessment; subsequently, we calculated the mean absolute difference (MAD) between the radiologists' BA predictions and ground-truth BA and evaluated the proportion of estimates for which the MAD exceeded one year. Additionally, we compared the radiologists' performance under conditions of early AI assistance with their performance under conditions of delayed AI assistance; the radiologists were allowed to reject AI interpretations.
The overall accuracy of senior, mid-level, and junior radiologists improved significantly with AI assistance than without AI assistance (MAD: 0.74 vs 0.46 years, p < 0.001; proportion of assessments for which MAD exceeded 1 year: 24.0% vs 8.4%, p < 0.001). The proportion of improved BA predictions with AI assistance (16.8%) was significantly higher than that of less accurate predictions with AI assistance (2.3%; p < 0.001). No consistent timing effect was observed between conditions of early and delayed AI assistance. Most disagreements between radiologists and AI occurred over images for patients aged ≤8 years. Senior radiologists had more disagreements than other radiologists.
The AI algorithm improved the BA assessment accuracy of radiologists with varying experience levels. Automation bias was prone to affect less experienced radiologists.
人工智能（AI）算法可以提供快速且精确的放射学骨龄（BA）评估。本研究评估了一种AI算法对放射科医生BA评估表现的影响，并评估了自动化偏差如何影响放射科医生。
在这项前瞻性随机交叉研究中，六名经验水平不同（高级、中级和初级）的放射科医生对一组200张标准BAX光片的测试集病例进行评估。测试集被平均分为两个子集：数据集A和B。每位放射科医生在无AI辅助（A - B -）和有AI辅助（A + B +）的情况下独立评估BA。我们将两位专家的评估均值作为准确性评估的金标准；随后，我们计算放射科医生的BA预测值与金标准BA之间的平均绝对差值（MAD），并评估MAD超过一年的估计比例。此外，我们比较了放射科医生在早期AI辅助条件下和延迟AI辅助条件下的表现；放射科医生可以拒绝AI的解读。
与无AI辅助相比，高级、中级和初级放射科医生在有AI辅助时的总体准确性显著提高（MAD：0.74岁对0.46岁，p < 0.001；MAD超过1年的评估比例：24.0%对8.4%，p < 0.001）。有AI辅助时BA预测改善的比例（16.8%）显著高于有AI辅助时预测准确性降低的比例（2.3%；p < 0.001）。在早期和延迟AI辅助条件之间未观察到一致的时间效应。放射科医生与AI之间的大多数分歧发生在年龄≤8岁患者的图像上。高级放射科医生比其他放射科医生有更多的分歧。
AI算法提高了不同经验水平放射科医生的BA评估准确性。自动化偏差容易影响经验较少的放射科医生。
