Lau Wilson, Aaltonen Laura, Gunn Martin, Yetisgen Meliha
Department of Biomedical and Health Informatics.
Department of Radiology.
AMIA Annu Symp Proc. 2022 Feb 21;2021:668-676. eCollection 2021.
Selecting radiology examination protocol is a repetitive, and time-consuming process. In this paper, we present a deep learning approach to automatically assign protocols to computed tomography examinations, by pre-training a domain-specific BERT model (BERT). To handle the high data imbalance across exam protocols, we used a knowledge distillation approach that up-sampled the minority classes through data augmentation. We compared classification performance of the described approach with n-gram models using Support Vector Machine (SVM), Gradient Boosting Machine (GBM), and Random Forest (RF) classifiers, as well as the BERT model. SVM, GBM and RF achieved macro-averaged F1 scores of 0.45, 0.45, and 0.6 while BERT and BERT achieved 0.61 and 0.63. Knowledge distillation boosted performance on the minority classes and achieved an F1 score of 0.66.
选择放射学检查方案是一个重复且耗时的过程。在本文中，我们提出了一种深度学习方法，通过预训练特定领域的BERT模型（BERT）来自动为计算机断层扫描检查分配方案。为了处理不同检查方案之间的数据高度不平衡问题，我们使用了一种知识蒸馏方法，通过数据增强对少数类别进行上采样。我们将所描述方法的分类性能与使用支持向量机（SVM）、梯度提升机（GBM）和随机森林（RF）分类器的n-gram模型以及BERT模型进行了比较。SVM、GBM和RF的宏平均F1分数分别为0.45、0.45和0.6，而BERT和BERT分别为0.61和0.63。知识蒸馏提高了少数类别的性能，F1分数达到了0.66。
