School of Computer Science and Information Engineering, The Catholic University of Korea, Bucheon, Republic of Korea.
Korea University Research Institute for Medical Bigdata Science, Korea University, Seoul, Republic of Korea.
Sci Rep. 2022 Aug 16;12(1):13847. doi: 10.1038/s41598-022-17806-8.
With advances in deep learning and natural language processing (NLP), the analysis of medical texts is becoming increasingly important. Nonetheless, despite the importance of processing medical texts, no research on Korean medical-specific language models has been conducted. The Korean medical text is highly difficult to analyze because of the agglutinative characteristics of the language, as well as the complex terminologies in the medical domain. To solve this problem, we collected a Korean medical corpus and used it to train the language models. In this paper, we present a Korean medical language model based on deep learning NLP. The model was trained using the pre-training framework of BERT for the medical context based on a state-of-the-art Korean language model. The pre-trained model showed increased accuracies of 0.147 and 0.148 for the masked language model with next sentence prediction. In the intrinsic evaluation, the next sentence prediction accuracy improved by 0.258, which is a remarkable enhancement. In addition, the extrinsic evaluation of Korean medical semantic textual similarity data showed a 0.046 increase in the Pearson correlation, and the evaluation for the Korean medical named entity recognition showed a 0.053 increase in the F1-score.
随着深度学习和自然语言处理（NLP）的进步，医学文本的分析变得越来越重要。尽管处理医学文本非常重要，但目前还没有针对韩国医学专用语言模型的研究。由于语言的粘性特征以及医学领域复杂的术语，韩国医学文本的分析非常困难。为了解决这个问题，我们收集了一个韩语医学语料库，并使用它来训练语言模型。在本文中，我们提出了一种基于深度学习 NLP 的韩语医学语言模型。该模型是基于一种最先进的韩语语言模型，使用 BERT 的医学上下文预训练框架进行训练的。预训练模型在带下一句预测的屏蔽语言模型中的准确率提高了 0.147 和 0.148。在内在评估中，下一句预测的准确率提高了 0.258，这是一个显著的提高。此外，对韩语医学语义文本相似性数据的外部评估显示 Pearson 相关性增加了 0.046，对韩语医学命名实体识别的评估显示 F1 分数增加了 0.053。
