Radiography and Diagnostic Imaging, School of Medicine, University College Dublin, Ireland; Society of Artificial Intelligence in Healthcare, Riyadh, Saudi Arabia.
Radiography and Diagnostic Imaging, School of Medicine, University College Dublin, Ireland.
Radiography (Lond). 2022 Aug;28(3):674-683. doi: 10.1016/j.radi.2022.05.005. Epub 2022 Jun 11.
Referrals vetting is a necessary daily task to ensure the appropriateness of radiology referrals. Vetting requires extensive clinical knowledge and may challenge those responsible. This study aims to develop AI models to automate the vetting process and to compare their performance with healthcare professionals.
1020 lumbar spine MRI referrals were collected retrospectively from two Irish hospitals. Three expert MRI radiographers classified the referrals into indicated or not indicated for scanning based on iRefer guidelines. The reference label for each referral was assigned based on the majority voting. The corpus was divided into two datasets, one for the models' development with 920 referrals, and one included 100 referrals used as a held-out for the final comparison of the AI models versus national and international MRI radiographers. Three traditional models were developed: SVM, LR, RF, and two deep neural models, including CNN and Bi-LSTM. For the traditional models, four vectorisation techniques applied: BoW, bigrams, trigrams, and TF-IDF. A textual data augmentation technique was applied to investigate the influence of data augmentation on the models' performances.
RF with BoW achieved the highest AUC reaching 0.99. CNN model outperformed Bi-LSTM with AUC = 0.98. With the augmented dataset, the performance significantly improved with an increase in F1 scores ranging from 1% to 7%. All models outperformed the national and international radiographers when compared on the hold-out dataset.
The models assigned the referrals' appropriateness with higher accuracies than the national and international radiographers. Applying data augmentation significantly improved the models' performances.
The outcomes suggest that the use of AI for checking referrals' eligibility could serve as a supporting tool to improve the referrals' management in radiology departments.
转诊审核是确保放射学转诊适当性的必要日常任务。审核需要广泛的临床知识，可能会对负责的人构成挑战。本研究旨在开发人工智能模型来自动化审核过程，并将其性能与医疗保健专业人员进行比较。
从爱尔兰的两家医院回顾性收集了 1020 例腰椎 MRI 转诊。三位经验丰富的 MRI 放射技师根据 iRefer 指南将转诊分为有扫描指征和无扫描指征。每个转诊的参考标签是根据多数投票分配的。语料库分为两个数据集，一个用于模型开发，包含 920 个转诊，另一个包含 100 个转诊，用于最终比较人工智能模型与国家和国际 MRI 放射技师的结果。开发了三种传统模型：SVM、LR、RF，以及两种深度神经网络模型，包括 CNN 和 Bi-LSTM。对于传统模型，应用了四种向量化技术：BoW、二元组、三元组和 TF-IDF。应用了文本数据扩充技术来研究数据扩充对模型性能的影响。
RF 与 BoW 结合的 AUC 最高，达到 0.99。CNN 模型的 AUC 优于 Bi-LSTM，为 0.98。在使用扩充数据集后，F1 评分从 1%到 7%的显著提高。在与预留数据集进行比较时，所有模型的表现均优于国家和国际放射技师。
与国家和国际放射技师相比，模型对转诊的适当性分配具有更高的准确性。应用数据扩充显著提高了模型的性能。
结果表明，使用人工智能检查转诊的资格可以作为放射科部门转诊管理的支持工具。
