School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, China.
School of Electrical Engineering, Tsinghua University, Beijing, China.
PLoS One. 2021 Sep 10;16(9):e0257092. doi: 10.1371/journal.pone.0257092. eCollection 2021.
Recent relation extraction models' architecture are evolved from the shallow neural networks to natural language model, such as convolutional neural networks or recurrent neural networks to Bert. However, these methods did not consider the semantic information in the sequence or the distance dependence problem, the internal semantic information may contain the useful knowledge which can help relation classification. Focus on these problems, this paper proposed a BERT-based relation classification method. Compare with the existing Bert-based architecture, the proposed model can obtain the internal semantic information between entity pair and solve the distance semantic dependence better. The pre-trained BERT model after fine tuning is used in this paper to abstract the semantic representation of sequence, then adopt the piecewise convolution to obtain semantic information which influence the extraction results. Compare with the existing methods, the proposed method can achieve a better accuracy on relational extraction task because of the internal semantic information extracted in the sequence. While, the generalization ability is still a problem that cannot be ignored, and the numbers of the relationships are difference between different categories. In this paper, the focal loss function is adopted to solve this problem by assigning a heavy weight to less number or hard classify categories. Finally, comparing with the existing methods, the F1 metric of the proposed method can reach a superior result 89.95% on the SemEval-2010 Task 8 dataset.
最近的关系抽取模型的架构是从浅层神经网络发展到自然语言模型，如卷积神经网络或循环神经网络到 Bert。然而，这些方法没有考虑序列中的语义信息或距离依赖问题，内部语义信息可能包含有助于关系分类的有用知识。针对这些问题，本文提出了一种基于 BERT 的关系分类方法。与现有的基于 Bert 的架构相比，所提出的模型可以获得实体对之间的内部语义信息，并更好地解决距离语义依赖问题。本文使用经过微调的预训练 BERT 模型来抽象序列的语义表示，然后采用分段卷积来获取影响提取结果的语义信息。与现有的方法相比，由于在序列中提取了内部语义信息，所提出的方法可以在关系抽取任务中实现更好的准确性。然而，泛化能力仍然是一个不容忽视的问题，不同类别之间的关系数量存在差异。在本文中，采用焦点损失函数通过为较少数量或难以分类的类别分配较大权重来解决这个问题。最后，与现有的方法相比，所提出的方法在 SemEval-2010 Task 8 数据集上的 F1 指标可以达到 89.95%的优异结果。
