School of Political Science and Public Management, WuHan University, Wuhan, China.
Department of Computer Science, Harbin Institute of Technology, Shenzhen, China.
BMC Med Inform Decis Mak. 2021 Jul 30;21(Suppl 2):94. doi: 10.1186/s12911-021-01442-9.
Text Matching (TM) is a fundamental task of natural language processing widely used in many application systems such as information retrieval, automatic question answering, machine translation, dialogue system, reading comprehension, etc. In recent years, a large number of deep learning neural networks have been applied to TM, and have refreshed benchmarks of TM repeatedly. Among the deep learning neural networks, convolutional neural network (CNN) is one of the most popular networks, which suffers from difficulties in dealing with small samples and keeping relative structures of features. In this paper, we propose a novel deep learning architecture based on capsule network for TM, called CapsTM, where capsule network is a new type of neural network architecture proposed to address some of the short comings of CNN and shows great potential in many tasks.
CapsTM is a five-layer neural network, including an input layer, a representation layer, an aggregation layer, a capsule layer and a prediction layer. In CapsTM, two pieces of text are first individually converted into sequences of embeddings and are further transformed by a highway network in the input layer. Then, Bidirectional Long Short-Term Memory (BiLSTM) is used to represent each piece of text and attention-based interaction matrix is used to represent interactive information of the two pieces of text in the representation layer. Subsequently, the two kinds of representations are fused together by BiLSTM in the aggregation layer, and are further represented with capsules (vectors) in the capsule layer. Finally, the prediction layer is a connected network used for classification. CapsTM is an extension of ESIM by adding a capsule layer before the prediction layer.
We construct a corpus of Chinese medical question matching, which contains 36,360 question pairs. This corpus is randomly split into three parts: a training set of 32,360 question pairs, a development set of 2000 question pairs and a test set of 2000 question pairs. On this corpus, we conduct a series of experiments to evaluate the proposed CapsTM and compare it with other state-of-the-art methods. CapsTM achieves the highest F-score of 0.8666.
The experimental results demonstrate that CapsTM is effective for Chinese medical question matching and outperforms other state-of-the-art methods for comparison.
文本匹配（TM）是自然语言处理的一项基本任务，广泛应用于信息检索、自动问答、机器翻译、对话系统、阅读理解等许多应用系统中。近年来，大量的深度学习神经网络被应用于 TM，并多次刷新 TM 的基准。在这些深度学习神经网络中，卷积神经网络（CNN）是最受欢迎的网络之一，但在处理小样本和保持特征的相对结构方面存在困难。在本文中，我们提出了一种基于胶囊网络的 TM 的新型深度学习架构，称为 CapsTM，其中胶囊网络是一种新的神经网络架构，旨在解决 CNN 的一些缺点，并在许多任务中显示出巨大的潜力。
CapsTM 是一个五层神经网络，包括输入层、表示层、聚合层、胶囊层和预测层。在 CapsTM 中，首先将两段文本分别转换为嵌入序列，并在输入层中通过高速公路网络进一步转换。然后，使用双向长短期记忆（BiLSTM）表示每段文本，并在表示层中使用基于注意力的交互矩阵表示两段文本的交互信息。随后，聚合层通过 BiLSTM 将两种表示融合在一起，并在胶囊层中进一步用胶囊（向量）表示。最后，预测层是一个用于分类的连接网络。CapsTM 是 ESIM 的扩展，在预测层之前添加了一个胶囊层。
我们构建了一个包含 36360 对问题的中文医疗问题匹配语料库。该语料库随机分为三部分：训练集包含 32360 对问题，开发集包含 2000 对问题，测试集包含 2000 对问题。在这个语料库上，我们进行了一系列实验来评估所提出的 CapsTM，并与其他最先进的方法进行比较。CapsTM 达到了 0.8666 的最高 F 值。
实验结果表明，CapsTM 对中文医疗问题匹配是有效的，并且优于其他最先进的方法进行比较。
