Ji Hongtao, Zhu Qiang, Ma Teng, Cheng Yun, Zhou Shuai, Ren Wei, Huang Huilian, He Wen, Ran Haitao, Ruan Litao, Guo Yanli, Tian Jiawei, Chen Wu, Chen Luzeng, Wang Zhiyuan, Zhou Qi, Niu Lijuan, Zhang Wei, Yang Ruimin, Chen Qin, Zhang Ruifang, Wang Hui, Li Li, Liu Minghui, Nie Fang, Zhou Aiyun
Department of Diagnostic Ultrasound, Beijing Tongren Hospital, Capital Medical University, Beijing, China.
Department of Ultrasonography, Beijing Tiantan Hospital, Capital Medical University, Beijing, China.
Quant Imaging Med Surg. 2023 Jun 1;13(6):3671-3687. doi: 10.21037/qims-22-1091. Epub 2023 Apr 28.
Significant differences exist in the classification outcomes for radiologists using ultrasonography-based Breast Imaging Reporting and Data Systems for diagnosing category 3-5 (BI-RADS 3-5) breast nodules, due to a lack of clear and distinguishing image features. Consequently, this retrospective study investigated the improvement of BI-RADS 3-5 classification consistency using a transformer-based computer-aided diagnosis (CAD) model.
Independently, 5 radiologists performed BI-RADS annotations on 21,332 breast ultrasonographic images collected from 3,978 female patients from 20 clinical centers in China. All images were divided into training, validation, testing, and sampling sets. The trained transformer-based CAD model was then used to classify test images, for which sensitivity (SEN), specificity (SPE), accuracy (ACC), area under the curve (AUC), and calibration curve were evaluated. Variations in these metrics among the 5 radiologists were analyzed by referencing BI-RADS classification results for the sampling test set provided by CAD to determine whether classification consistency (the k value), SEN, SPE, and ACC could be improved.
After the training set (11,238 images) and validation set (2,996 images) were learned by the CAD model, the classification ACC of the CAD model applied to the test set (7,098 images) was 94.89% in category 3, 96.90% in category 4A, 95.49% in category 4B, 92.28% in category 4C, and 95.45% in category 5 nodules. Based on pathological results, the AUC of the CAD model was 0.924 and the predicted probability of CAD was a little higher than the actual probability in the calibration curve. After referencing BI-RADS classification results, the adjustments were made to 1,583 nodules, of which 905 were classified to a lower category and 678 to a higher category in the sampling test set. As a result, the ACC (72.41-82.65%), SEN (32.73-56.98%), and SPE (82.46-89.26%) of the classification by each radiologist were significantly improved on average, with the consistency (k values) in almost all of them increasing to >0.6.
The radiologist's classification consistency was markedly improved with almost all the k values increasing by a value greater than 0.6, and the diagnostic efficiency was also improved by approximately 24% (32.73% to 56.98%) and 7% (82.46% to 89.26%) for SEN and SPE, respectively, of the total classification on average. The transformer-based CAD model can help to improve the radiologist's diagnostic efficacy and consistency with others in the classification of BI-RADS 3-5 nodules.
由于缺乏清晰且有区分度的图像特征，放射科医生使用基于超声的乳腺影像报告和数据系统（BI-RADS 3-5）对乳腺结节进行诊断分类时，分类结果存在显著差异。因此，这项回顾性研究调查了使用基于Transformer的计算机辅助诊断（CAD）模型对BI-RADS 3-5分类一致性的改善情况。
5名放射科医生独立地对从中国20个临床中心的3978名女性患者收集的21332幅乳腺超声图像进行BI-RADS标注。所有图像被分为训练集、验证集、测试集和采样集。然后使用经过训练的基于Transformer的CAD模型对测试图像进行分类，并评估其灵敏度（SEN）、特异度（SPE）、准确度（ACC）、曲线下面积（AUC）和校准曲线。通过参考CAD提供的采样测试集的BI-RADS分类结果，分析这5名放射科医生在这些指标上的差异，以确定分类一致性（k值）、SEN、SPE和ACC是否能够得到改善。
在CAD模型学习了训练集（11238幅图像）和验证集（2996幅图像）后，应用于测试集（7098幅图像）的CAD模型的分类ACC在3类结节中为94.89%，4A类为96.90%，4B类为95.49%，4C类为92.28%，5类结节为95.45%。基于病理结果，CAD模型的AUC为0.924，在校准曲线中CAD的预测概率略高于实际概率。参考BI-RADS分类结果后，对1583个结节进行了调整，其中在采样测试集中905个被分类到较低类别，678个被分类到较高类别。结果，每位放射科医生分类的ACC（72.41%-82.65%）、SEN（32.73%-56.98%）和SPE（82.46%-89.26%）平均显著提高，几乎所有放射科医生的一致性（k值）都增加到>0.6。
放射科医生的分类一致性显著提高，几乎所有k值都增加了大于0.6的值，并且诊断效率平均也分别提高了约24%（SEN从32.73%提高到56.98%）和7%（SPE从82.46%提高到89.26%）。基于Transformer的CAD模型有助于提高放射科医生在BI-RADS 3-5结节分类中的诊断效能和与他人的一致性。
