Pastor Maxime, Dabli Djamel, Lonjon Raphaël, Serrand Chris, Snene Fehmi, Trad Fayssal, de Oliveira Fabien, Beregi Jean-Paul, Greffier Joël
IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France.
IMAGINE UR UM 103, Montpellier University, Department of Medical Imaging, Nîmes University Hospital, 30900 Nîmes, France.
Diagn Interv Imaging. 2025 Jan;106(1):22-27. doi: 10.1016/j.diii.2024.09.004. Epub 2024 Sep 19.
The purpose of this study was to compare the diagnostic performance of an artificial intelligence (AI) solution for the detection of fractures of pelvic, proximal femur or extremity fractures in adults with radiologist interpretation of radiographs, using standard dose CT examination as the standard of reference.
This retrospective study included 94 adult patients with suspected bone fractures who underwent a standard dose CT examination and radiographs of the pelvis and/or hip and extremities at our institution between January 2022 and August 2023. For all patients, an AI solution was used retrospectively on the radiographs to detect and localize bone fractures of the pelvis and/or hip and extremities. Results of the AI solution were compared to the reading of each radiograph by a radiologist using McNemar test. The results of standard dose CT examination as interpreted by a senior radiologist were used as the standard of reference.
A total of 94 patients (63 women; mean age, 56.4 ± 22.5 [standard deviation] years) were included. Forty-seven patients had at least one fracture, and a total of 71 fractures were deemed present using the standard of reference (25 hand/wrist, 16 pelvis, 30 foot/ankle). Using the standard of reference, the analysis of radiographs by the AI solution resulted in 58 true positive, 13 false negative, 33 true negative and 15 false positive findings, yielding 82 % sensitivity (58/71; 95 % confidence interval [CI]: 71-89 %), 69 % specificity (33/48; 95 % CI: 55-80 %), and 76 % accuracy (91/119; 95 % CI: 69-84 %). Using the standard of reference, the reading of the radiologist resulted in 65 true positive, 6 false negative, 42 true negative and 6 false positive findings, yielding 92 % sensitivity (65/71; 95 % CI: 82-96 %), 88 % specificity (42/48; 95 % CI: 75-94 %), and 90 % accuracy (107/119; 95 % CI: 85-95 %). The radiologist outperformed the AI solution in terms of sensitivity (P = 0.045), specificity (P = 0.016), and accuracy (P < 0.001).
In this study, the radiologist outperformed the AI solution for the diagnosis of pelvic, hip and extremity fractures of the using radiographs. This raises the question of whether a strong standard of reference for evaluating AI solutions should be used in future studies comparing AI and human reading in fracture detection using radiographs.
本研究旨在比较人工智能（AI）解决方案与放射科医生对成人骨盆、股骨近端或四肢骨折的X线片解读在检测骨折方面的诊断性能，以标准剂量CT检查作为参考标准。
这项回顾性研究纳入了94例疑似骨折的成年患者，这些患者于2022年1月至2023年8月在我们机构接受了标准剂量CT检查以及骨盆和/或髋部及四肢的X线片检查。对于所有患者，对X线片进行回顾性使用AI解决方案来检测和定位骨盆和/或髋部及四肢的骨折。将AI解决方案的结果与放射科医生对每张X线片的读片结果进行McNemar检验比较。由资深放射科医生解读的标准剂量CT检查结果用作参考标准。
共纳入94例患者（63例女性；平均年龄56.4±22.5[标准差]岁）。47例患者至少有一处骨折，根据参考标准共发现71处骨折（25处手部/腕部，16处骨盆，30处足部/踝部）。根据参考标准，AI解决方案对X线片的分析得出58例假阳性、13例假阴性、33例真阴性和15例真阳性结果，灵敏度为82%（58/71；95%置信区间[CI]：71 - 89%），特异度为69%（33/48；95%CI：55 - 80%），准确率为76%（91/119；95%CI：69 - 84%）。根据参考标准，放射科医生的读片结果得出65例假阳性、6例假阴性、42例真阴性和6例真阳性结果，灵敏度为92%（65/71；95%CI：82 - 96%），特异度为88%（42/48；95%CI：75 - 94%），准确率为90%（107/119；95%CI：85 - 95%）。放射科医生在灵敏度（P = 0.045）、特异度（P = 0.016）和准确率（P < 0.001）方面优于AI解决方案。
在本研究中，放射科医生在使用X线片诊断骨盆、髋部和四肢骨折方面优于AI解决方案。这就提出了一个问题，即在未来比较AI和人类在使用X线片检测骨折方面的读片结果的研究中，是否应使用更强的参考标准来评估AI解决方案。
