Sui Dianbo, Zeng Xiangrong, Chen Yubo, Liu Kang, Zhao Jun
IEEE Trans Neural Netw Learn Syst. 2024 Sep;35(9):12784-12795. doi: 10.1109/TNNLS.2023.3264735. Epub 2024 Sep 3.
Joint entity and relation extraction is an important task in natural language processing, which aims to extract all relational triples mentioned in a given sentence. In essence, the relational triples mentioned in a sentence are in the form of a set, which has no intrinsic order between elements and exhibits the permutation invariant feature. However, previous seq2seq-based models require sorting the set of relational triples into a sequence beforehand with some heuristic global rules, which destroys the natural set structure. In order to break this bottleneck, we treat joint entity and relation extraction as a direct set prediction problem, so that the extraction model is not burdened with predicting the order of multiple triples. To solve this set prediction problem, we propose networks featured by transformers with non-autoregressive parallel decoding. In contrast to autoregressive approaches that generate triples one by one in a specific order, the proposed networks are able to directly output the final set of relational triples in one shot. Furthermore, we also design a set-based loss that forces unique predictions through bipartite matching. Compared with cross-entropy loss that highly penalizes small shifts in triple order, the proposed bipartite matching loss is invariant to any permutation of predictions; thus, it can provide the proposed networks with a more accurate training signal by ignoring triple order and focusing on relation types and entities. Various experiments on two benchmark datasets demonstrate that our proposed model significantly outperforms the current state-of-the-art (SoTA) models. Training code and trained models are now publicly available at https://github.com/DianboWork/SPN4RE.
联合实体与关系提取是自然语言处理中的一项重要任务，其目的是提取给定句子中提到的所有关系三元组。本质上，句子中提到的关系三元组呈集合形式，其元素之间没有内在顺序，具有排列不变性特征。然而，先前基于序列到序列（seq2seq）的模型需要使用一些启发式全局规则预先将关系三元组集合排序成一个序列，这破坏了自然的集合结构。为了突破这一瓶颈，我们将联合实体与关系提取视为一个直接的集合预测问题，从而使提取模型无需承担预测多个三元组顺序的负担。为了解决这个集合预测问题，我们提出了以具有非自回归并行解码的变换器为特征的网络。与按特定顺序逐个生成三元组的自回归方法不同，所提出的网络能够一次性直接输出关系三元组的最终集合。此外，我们还设计了一种基于集合的损失函数，通过二分匹配强制进行唯一预测。与对三元组顺序的微小变化进行高度惩罚的交叉熵损失不同，所提出的二分匹配损失对预测的任何排列都是不变的；因此，它可以通过忽略三元组顺序并专注于关系类型和实体，为所提出的网络提供更准确的训练信号。在两个基准数据集上进行的各种实验表明，我们提出的模型显著优于当前的最先进（SoTA）模型。训练代码和训练好的模型现在可在https://github.com/DianboWork/SPN4RE上公开获取。
