Hermans Sebastiaan, Hu Zixuan, Ball Robyn L, Lin Hui Ming, Prevedello Luciano M, Berger Ferco H, Yusuf Ibrahim, Rudie Jeffrey D, Vazirabad Maryam, Flanders Adam E, Shih George, Mongan John, Nicolaou Savvas, Marinelli Brett S, Davis Melissa A, Magudia Kirti, Sejdić Ervin, Colak Errol
From the Department of Medical Imaging, St Michael's Hospital, Unity Health Toronto, 30 Bond St, Toronto, ON, Canada M5B 1W8 (S.H., Z.H., H.M.L., I.Y., E.C.); Edward S. Rogers Department of Electrical and Computer Engineering, University of Toronto, Toronto, Ontario, Canada (Z.H., E.S.); The Jackson Laboratory, Bar Harbor, Me (R.L.B.); Department of Radiology, The Ohio State University, Columbus, Ohio (L.M.P.); Department of Medical Imaging, Sunnybrook Health Sciences Centre, University of Toronto, Toronto, Ontario, Canada (F.H.B.); Department of Radiology, Scripps Clinic Medical Group and University of California San Diego, San Diego, Calif (J.D.R.); Radiological Society of North America, Oak Brook, Ill (M.V.); Department of Radiology, Thomas Jefferson University, Philadelphia, Pa (A.E.F.); Department of Radiology, Weill Cornell Medicine, New York, NY (G.S.); Department of Radiology and Biomedical Imaging, University of California San Francisco, San Francisco, Calif (J.M.), Department of Radiology, Vancouver General Hospital, Vancouver, Canada (S.N.); Department of Radiology, Memorial Sloan-Kettering Cancer Center, New York, NY (B.S.M.); Department of Radiology and Biomedical Imaging, Yale University School of Medicine, New Haven, Conn (M.A.D.); Duke University School of Medicine, Durham, NC (K.M.); North York General Hospital, Toronto, Ontario, Canada (E.S.); and Department of Medical Imaging, University of Toronto, Toronto, Ontario, Canada (E.C.).
Radiol Artif Intell. 2025 Jan;7(1):e240334. doi: 10.1148/ryai.240334.
Purpose To evaluate the performance of the winning machine learning models from the 2023 RSNA Abdominal Trauma Detection AI Challenge. Materials and Methods The competition was hosted on Kaggle and took place between July 26 and October 15, 2023. The multicenter competition dataset consisted of 4274 abdominal trauma CT scans, in which solid organs (liver, spleen, and kidneys) were annotated as healthy, low-grade, or high-grade injury. Studies were labeled as positive or negative for the presence of bowel and mesenteric injury and active extravasation. In this study, performances of the eight award-winning models were retrospectively assessed and compared using various metrics, including the area under the receiver operating characteristic curve (AUC), for each injury category. The reported mean values of these metrics were calculated by averaging the performance across all models for each specified injury type. Results The models exhibited strong performance in detecting solid organ injuries, particularly high-grade injuries. For binary detection of injuries, the models demonstrated mean AUC values of 0.92 (range, 0.90-0.94) for liver, 0.91 (range, 0.87-0.93) for splenic, and 0.94 (range, 0.93-0.95) for kidney injuries. The models achieved mean AUC values of 0.98 (range, 0.96-0.98) for high-grade liver, 0.98 (range, 0.97-0.99) for high-grade splenic, and 0.98 (range, 0.97-0.98) for high-grade kidney injuries. For the detection of bowel and mesenteric injuries and active extravasation, the models demonstrated mean AUC values of 0.85 (range, 0.74-0.93) and 0.85 (range, 0.79-0.89), respectively. Conclusion The award-winning models from the artificial intelligence challenge demonstrated strong performance in the detection of traumatic abdominal injuries on CT scans, particularly high-grade injuries. These models may serve as a performance baseline for future investigations and algorithms. Abdominal Trauma, CT, American Association for the Surgery of Trauma, Machine Learning, Artificial Intelligence © RSNA, 2024.
目的 评估在2023年RSNA腹部创伤检测人工智能挑战赛中获胜的机器学习模型的性能。材料与方法 该竞赛在Kaggle平台上举办，于2023年7月26日至10月15日举行。多中心竞赛数据集包含4274例腹部创伤CT扫描，其中实性器官（肝脏、脾脏和肾脏）被标注为健康、低度损伤或高度损伤。研究被标记为存在或不存在肠管和肠系膜损伤以及活动性出血。在本研究中，回顾性评估并比较了八个获奖模型在每个损伤类别的各种指标表现，包括受试者操作特征曲线下面积（AUC）。这些指标的报告均值是通过对每种特定损伤类型的所有模型性能进行平均计算得出的。结果 这些模型在检测实性器官损伤，尤其是高度损伤方面表现出色。对于损伤的二元检测，模型在肝脏损伤检测中的平均AUC值为0.92（范围0.90 - 0.94），脾脏损伤为0.91（范围0.87 - 0.93），肾脏损伤为0.94（范围0.93 - 0.95）。对于高度肝脏损伤，模型的平均AUC值为0.98（范围0.96 - 0.98），高度脾脏损伤为0.98（范围0.97 - 0.99），高度肾脏损伤为0.98（范围0.97 - 0.98）。对于肠管和肠系膜损伤以及活动性出血的检测，模型的平均AUC值分别为0.85（范围0.74 - 0.93）和0.85（范围0.79 - 0.89）。结论 在人工智能挑战赛中获奖的模型在CT扫描检测创伤性腹部损伤方面表现出色，尤其是高度损伤。这些模型可作为未来研究和算法的性能基线。腹部创伤、CT、美国创伤外科学会、机器学习、人工智能 © RSNA，2024年
