Goodwin Travis R, Demner-Fushman Dina
U.S. National Library of Medicine, National Institutes of Health.
Proc Conf Empir Methods Nat Lang Process. 2020 Nov;2020:56-63. doi: 10.18653/v1/2020.deelio-1.7.
Deep neural networks have demonstrated high performance on many natural language processing (NLP) tasks that can be answered directly from text, and have struggled to solve NLP tasks requiring external (e.g., world) knowledge. In this paper, we present OSCR (Ontology-based Semantic Composition Regularization), a method for injecting task-agnostic knowledge from an Ontology or knowledge graph into a neural network during pre-training. We evaluated the performance of BERT pre-trained on Wikipedia with and without OSCR by measuring the performance when fine-tuning on two question answering tasks involving world knowledge and causal reasoning and one requiring domain (healthcare) knowledge and obtained 33.3 %, 18.6 %, and 4 % improved accuracy compared to pre-training BERT without OSCR.
深度神经网络在许多可直接从文本回答的自然语言处理（NLP）任务上已展现出高性能，但在解决需要外部（如世界）知识的NLP任务时却面临困难。在本文中，我们提出了基于本体的语义组合正则化（OSCR）方法，这是一种在预训练期间将来自本体或知识图谱的与任务无关的知识注入神经网络的方法。我们通过在两个涉及世界知识和因果推理以及一个需要领域（医疗保健）知识的问答任务上进行微调时测量性能，评估了在有和没有OSCR的情况下在维基百科上预训练的BERT的性能，与没有OSCR预训练的BERT相比，准确率分别提高了33.3%、18.6%和4%。
