Division of Diagnostic Imaging, Sheba Medical Center, Sackler Faculty of Medicine, Tel Aviv University, Derech Sheba St 2, Ramat Gan, Israel.
DeepVision Lab, Sheba Medical Center, Ramat Gan, Israel.
Neuroradiology. 2020 Oct;62(10):1247-1256. doi: 10.1007/s00234-020-02420-0. Epub 2020 Apr 25.
Natural language processing (NLP) can be used for automatic flagging of radiology reports. We assessed deep learning models for classifying non-English head CT reports.
We retrospectively collected head CT reports (2011-2018). Reports were signed in Hebrew. Emergency department (ED) reports of adult patients from January to February for each year (2013-2018) were manually labeled. All other reports were used to pre-train an embedding layer. We explored two use cases: (1) general labeling use case, in which reports were labeled as normal vs. pathological; (2) specific labeling use case, in which reports were labeled as with and without intra-cranial hemorrhage. We tested long short-term memory (LSTM) and LSTM-attention (LSTM-ATN) networks for classifying reports. We also evaluated the improvement of adding Word2Vec word embedding. Deep learning models were compared with a bag-of-words (BOW) model.
We retrieved 176,988 head CT reports for pre-training. We manually labeled 7784 reports as normal (46.3%) or pathological (53.7%), and 7.1% with intra-cranial hemorrhage. For the general labeling, LSTM-ATN-Word2Vec showed the best results (AUC = 0.967 ± 0.006, accuracy 90.8% ± 0.01). For the specific labeling, all methods showed similar accuracies between 95.0 and 95.9%. Both LSTM-ATN-Word2Vec and BOW had the highest AUC (0.970).
For a general use case, word embedding using a large cohort of non-English head CT reports and ATN improves NLP performance. For a more specific task, BOW and deep learning showed similar results. Models should be explored and tailored to the NLP task.
自然语言处理（NLP）可用于自动标记放射学报告。我们评估了深度学习模型在分类非英语头部 CT 报告中的性能。
我们回顾性地收集了头部 CT 报告（2011-2018 年）。报告以希伯来语签署。每年 1 月至 2 月（2013-2018 年）从急诊科（ED）收集成年患者的 CT 报告，并进行手动标记。所有其他报告均用于预训练嵌入层。我们探索了两种用例：（1）通用标记用例，其中报告被标记为正常与异常；（2）特定标记用例，其中报告被标记为有或无颅内出血。我们测试了长短期记忆（LSTM）和 LSTM-注意力（LSTM-ATN）网络来分类报告。我们还评估了添加 Word2Vec 词嵌入的改进效果。深度学习模型与词袋（BOW）模型进行了比较。
我们为预训练检索了 176988 个头部 CT 报告。我们手动标记了 7784 个报告为正常（46.3%）或异常（53.7%），7.1%有颅内出血。对于通用标记，LSTM-ATN-Word2Vec 表现最佳（AUC=0.967±0.006，准确率 90.8%±0.01）。对于特定标记，所有方法的准确率在 95.0%至 95.9%之间相似。LSTM-ATN-Word2Vec 和 BOW 均具有最高 AUC（0.970）。
对于通用用例，使用大量非英语头部 CT 报告和 ATN 的词嵌入可以提高 NLP 性能。对于更具体的任务，BOW 和深度学习的表现相似。应根据 NLP 任务探索和定制模型。
