Ke Xianxin, Hu Ping, Yang Chenghao, Zhang Renbao
School of Mechanical and Electrical Engineering and Automation, Shanghai University, Shanghai 200444, China.
Micromachines (Basel). 2022 Feb 23;13(3):355. doi: 10.3390/mi13030355.
During multi-turn dialogue, with the increase in dialogue turns, the difficulty of intention recognition and the generation of the following sentence reply become more and more difficult. This paper mainly optimizes the context information extraction ability of the Seq2Seq Encoder in multi-turn dialogue modeling. We fuse the historical dialogue information and the current input statement information in the encoder to capture the context dialogue information better. Therefore, we propose a BERT-based fusion encoder ProBERT-To-GUR (PBTG) and an enhanced ELMO model 3-ELMO-Attention-GRU (3EAG). The two models mainly enhance the contextual information extraction capability of multi-turn dialogue. To verify the effectiveness of the two proposed models, we demonstrate the effectiveness of our model by combining data based on the LCCC-large multi-turn dialogue dataset and the Naturalconv multi-turn dataset. The experimental comparison results show that, in the multi-turn dialogue experiments of the open domain and fixed topic, the two Seq2Seq coding models proposed are significantly improved compared with the current state-of-the-art models. For specified topic multi-turn dialogue, the 3EAG model has the average BLEU value reaches the optimal 32.4, which achieves the best language generation effect, and the BLEU value in the actual dialogue verification experiment also surpasses 31.8. for open-domain multi-turn dialogue. The average BLEU value of the PBTG model reaches 31.8, the optimal 31.8 achieves the best language generation effect, and the BLEU value in the actual dialogue verification experiment surpasses 31.2. So, the 3EAG model is more suitable for fixed-topic multi-turn dialogues for the two tasks. The PBTG model is more muscular in open-domain multi-turn dialogue tasks; therefore, our model is significant for promoting multi-turn dialogue research.
在多轮对话中，随着对话轮数的增加，意图识别的难度以及生成后续句子回复变得越来越困难。本文主要优化了多轮对话建模中Seq2Seq编码器的上下文信息提取能力。我们在编码器中融合历史对话信息和当前输入语句信息，以更好地捕捉上下文对话信息。因此，我们提出了基于BERT的融合编码器ProBERT-To-GUR（PBTG）和增强的ELMO模型3-ELMO-Attention-GRU（3EAG）。这两个模型主要增强了多轮对话的上下文信息提取能力。为了验证所提出的两个模型的有效性，我们通过结合基于LCCC-large多轮对话数据集和Naturalconv多轮数据集的数据来证明我们模型的有效性。实验比较结果表明，在开放域和固定主题的多轮对话实验中，所提出的两个Seq2Seq编码模型与当前最先进的模型相比有显著改进。对于指定主题的多轮对话，3EAG模型的平均BLEU值达到最优的32.4，实现了最佳的语言生成效果，并且在实际对话验证实验中的BLEU值也超过了31.8。对于开放域多轮对话，PBTG模型的平均BLEU值达到31.8，最优的31.8实现了最佳的语言生成效果，并且在实际对话验证实验中的BLEU值超过了31.2。所以，对于这两项任务，3EAG模型更适合固定主题的多轮对话。PBTG模型在开放域多轮对话任务中更有优势；因此，我们的模型对推动多轮对话研究具有重要意义。
