Liu Hao, Perl Yehoshua, Geller James
Dept of Computer Science, NJIT, Newark, NJ, USA.
AMIA Annu Symp Proc. 2020 Mar 4;2019:1129-1138. eCollection 2019.
With advances in Machine Learning (ML), neural network-based methods, such as Convolutional/Recurrent Neural Networks, have been proposed to assist terminology curators in the development and maintenance of terminologies. Bidirectional Encoder Representations from Transformers (BERT), a new language representation model, obtains state-of-the-art results on a wide array of general English NLP tasks. We explore BERT's applicability to medical terminology-related tasks. Utilizing the "next sentence prediction" capability of BERT, we show that the Fine-tuning strategy of Transfer Learning (TL) from the BERT model can address a challenging problem in automatic terminology enrichment - insertion of new concepts. Adding a pre-training strategy enhances the results. We apply our strategies to the two largest hierarchies of SNOMED CT, with one release as training data and the following release as test data. The performance of the combined two proposed TL models achieves an average F1 score of 0.85 and 0.86 for the two hierarchies, respectively.
随着机器学习（ML）的发展，已经提出了基于神经网络的方法，如卷积/循环神经网络，以协助术语管理人员进行术语的开发和维护。来自变换器的双向编码器表示（BERT）是一种新的语言表示模型，在一系列通用英语自然语言处理任务中取得了领先成果。我们探索BERT在医学术语相关任务中的适用性。利用BERT的“下一句预测”能力，我们表明从BERT模型进行迁移学习（TL）的微调策略可以解决自动术语丰富中的一个具有挑战性的问题——插入新概念。添加预训练策略可提高结果。我们将我们的策略应用于SNOMED CT的两个最大层次结构，将一个版本作为训练数据，下一个版本作为测试数据。所提出的两个组合TL模型在两个层次结构上的性能分别达到了平均F1分数0.85和0.86。
