Li Jing, Zhang Dezheng, Wulamu Aziguli
School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China.
Beijing Key Laboratory of Knowledge Engineering for Materials Science, University of Science and Technology Beijing, Beijing 100083, China.
Entropy (Basel). 2022 Apr 23;24(5):590. doi: 10.3390/e24050590.
At present, short text classification is a hot topic in the area of natural language processing. Due to the sparseness and irregularity of short text, the task of short text classification still faces great challenges. In this paper, we propose a new classification model from the aspects of short text representation, global feature extraction and local feature extraction. We use convolutional networks to extract shallow features from short text vectorization, and introduce a multi-level semantic extraction framework. It uses BiLSTM as the encoding layer while the attention mechanism and normalization are used as the interaction layer. Finally, we concatenate the convolution feature vector and semantic results of the semantic framework. After several rounds of feature integration, the framework improves the quality of the feature representation. Combined with the capsule network, we obtain high-level local information by dynamic routing and then squash them. In addition, we explore the optimal depth of semantic feature extraction for short text based on a multi-level semantic framework. We utilized four benchmark datasets to demonstrate that our model provides comparable results. The experimental results show that the accuracy of SUBJ, TREC, MR and ProcCons are 93.8%, 91.94%, 82.81% and 98.43%, respectively, which verifies that our model has greatly improves classification accuracy and model robustness.
目前，短文本分类是自然语言处理领域的一个热门话题。由于短文本的稀疏性和不规则性，短文本分类任务仍然面临巨大挑战。在本文中，我们从短文本表示、全局特征提取和局部特征提取等方面提出了一种新的分类模型。我们使用卷积网络从短文本向量化中提取浅层特征，并引入了一个多层次语义提取框架。它使用双向长短期记忆网络（BiLSTM）作为编码层，同时使用注意力机制和归一化作为交互层。最后，我们将卷积特征向量和语义框架的语义结果连接起来。经过几轮特征整合，该框架提高了特征表示的质量。结合胶囊网络，我们通过动态路由获得高级局部信息，然后对其进行压缩。此外，我们基于多层次语义框架探索了短文本语义特征提取的最佳深度。我们利用四个基准数据集来证明我们的模型提供了可比的结果。实验结果表明，SUBJ、TREC、MR和ProcCons的准确率分别为93.8%、91.94%、82.81%和98.43%，这验证了我们的模型大大提高了分类准确率和模型鲁棒性。
