From the Department of Biomedical Imaging and Image-guided Therapy, Division of Molecular and Gender Imaging, Medical University of Vienna, Austria (D.R.); Department of Radiology, Breast Imaging Service, Memorial Sloan-Kettering Cancer Center, New York, NY (R.L.G., J.T.); Center for Medical Physics and Biomedical Engineering, Medical University Vienna, Vienna, Austria (F.S., J.H.); St Francis Hospital Vienna, Vienna, Austria (A.R.); Sigmund Freud University Medical School, Vienna, Austria (A.R.); and Department of Radiology, Division of Breast Imaging, Columbia University Irving Medical Center, 161 Fort Washington Ave, New York, NY 10032 (K.P.).
Radiol Imaging Cancer. 2024 Jul;6(4):e230149. doi: 10.1148/rycan.230149.
Purpose To compare two deep learning-based commercially available artificial intelligence (AI) systems for mammography with digital breast tomosynthesis (DBT) and benchmark them against the performance of radiologists. Materials and Methods This retrospective study included consecutive asymptomatic patients who underwent mammography with DBT (2019-2020). Two AI systems (Transpara 1.7.0 and ProFound AI 3.0) were used to evaluate the DBT examinations. The systems were compared using receiver operating characteristic (ROC) analysis to calculate the area under the ROC curve (AUC) for detecting malignancy overall and within subgroups based on mammographic breast density. Breast Imaging Reporting and Data System results obtained from standard-of-care human double-reading were compared against AI results with use of the DeLong test. Results Of 419 female patients (median age, 60 years [IQR, 52-70 years]) included, 58 had histologically proven breast cancer. The AUC was 0.86 (95% CI: 0.85, 0.91), 0.93 (95% CI: 0.90, 0.95), and 0.98 (95% CI: 0.96, 0.99) for Transpara, ProFound AI, and human double-reading, respectively. For Transpara, a rule-out criterion of score 7 or lower yielded 100% (95% CI: 94.2, 100.0) sensitivity and 60.9% (95% CI: 55.7, 66.0) specificity. The rule-in criterion of higher than score 9 yielded 96.6% sensitivity (95% CI: 88.1, 99.6) and 78.1% specificity (95% CI: 73.8, 82.5). For ProFound AI, a rule-out criterion of lower than score 51 yielded 100% sensitivity (95% CI: 93.8, 100) and 67.0% specificity (95% CI: 62.2, 72.1). The rule-in criterion of higher than score 69 yielded 93.1% (95% CI: 83.3, 98.1) sensitivity and 82.0% (95% CI: 77.9, 86.1) specificity. Conclusion Both AI systems showed high performance in breast cancer detection but lower performance compared with human double-reading. Mammography, Breast, Oncology, Artificial Intelligence, Deep Learning, Digital Breast Tomosynthesis © RSNA, 2024.
目的 比较两种基于深度学习的商用人工智能（AI）系统在乳腺 X 线摄影与数字乳腺断层合成（DBT）中的表现，并与放射科医生的表现进行基准比较。
材料与方法 本回顾性研究纳入了 2019 年至 2020 年期间接受乳腺 X 线摄影与 DBT 检查的连续无症状患者。使用两种 AI 系统（Transpara 1.7.0 和 ProFound AI 3.0）来评估 DBT 检查。使用受试者工作特征（ROC）分析比较系统，计算 ROC 曲线下面积（AUC），以评估总体及基于乳腺 X 线摄影乳腺密度的亚组中恶性肿瘤的检出率。使用 DeLong 检验将 AI 结果与标准护理下的人类双读片的乳腺影像报告和数据系统（BI-RADS）结果进行比较。
结果 在 419 名女性患者中（中位年龄 60 岁[IQR，52-70 岁]），58 名患者经组织学证实患有乳腺癌。Transpara、ProFound AI 和人类双读片的 AUC 分别为 0.86（95%CI：0.85，0.91）、0.93（95%CI：0.90，0.95）和 0.98（95%CI：0.96，0.99）。对于 Transpara，评分 7 或更低的排除标准可实现 100%（95%CI：94.2，100.0）的敏感性和 60.9%（95%CI：55.7，66.0）的特异性。评分高于 9 的纳入标准可实现 96.6%（95%CI：88.1，99.6）的敏感性和 78.1%（95%CI：73.8，82.5）的特异性。对于 ProFound AI，评分低于 51 的排除标准可实现 100%（95%CI：93.8，100）的敏感性和 67.0%（95%CI：62.2，72.1）的特异性。评分高于 69 的纳入标准可实现 93.1%（95%CI：83.3，98.1）的敏感性和 82.0%（95%CI：77.9，86.1）的特异性。
结论 两种 AI 系统在乳腺癌检测中均表现出较高的性能，但与人类双读片相比性能较低。
