Liu Qi, Xiao Kejing, Qian Zhaopeng
School of Information Engineering, Beijing Institute of Graphic Communication, Beijing, China.
School of Computer and Artificial Intelligence, Beijing Technology and Business University, Beijing, China.
Sci Rep. 2025 Mar 18;15(1):9333. doi: 10.1038/s41598-025-90864-w.
Text classification is an important task in the field of natural language processing, aiming to automatically assign text data to predefined categories. The BertGCN model combines the advantages from both BERT and GCN, enabling it to effectively handle text data for classification. However, there are still some limitations when it comes to handling complex text classification tasks. BERT processes sequence information in segments and cannot directly capture long-distance dependencies across segments, which is a limitation when dealing with long sequences. GCN tends to suffer from over-smoothing problem in deep networks, leading to information loss. To overcome these limitations, we propose the XLG-Net model, which integrates XLNet and GCNII to enhance text classification performance. XLNet employs permutation language modeling and improvements of the Transformer-XL architecture, not only improving the ability to capture long-distance dependencies but also enhancing the model's understanding of complex language structures. Additionally, we introduce GCNII to overcome the over-smoothing problem in GCN. GCNII effectively retains the initial features of nodes by incorporating initial residual connections and identity mapping mechanisms, ensuring effective information transmission even in deep networks. Furthermore, to achieve excellent performance on both long and short texts, we apply the design philosophy of DoubleMix to the XLNet model, using a hybrid approach of mixing hidden states improves the model's accuracy and robustness. Experimental results demonstrate that the XLG-Net model achieves significant performance improvements on four benchmark text classification datasets, validating the model's effectiveness on complex text classification tasks.
文本分类是自然语言处理领域中的一项重要任务，旨在将文本数据自动分配到预定义的类别中。BertGCN模型结合了BERT和GCN的优点，使其能够有效地处理文本数据进行分类。然而，在处理复杂的文本分类任务时仍存在一些局限性。BERT按段处理序列信息，无法直接捕捉跨段的长距离依赖关系，这在处理长序列时是一个限制。GCN在深度网络中容易出现过平滑问题，导致信息丢失。为了克服这些局限性，我们提出了XLG-Net模型，该模型集成了XLNet和GCNII以提高文本分类性能。XLNet采用排列语言建模和对Transformer-XL架构的改进，不仅提高了捕捉长距离依赖关系的能力，还增强了模型对复杂语言结构的理解。此外，我们引入GCNII来克服GCN中的过平滑问题。GCNII通过合并初始残差连接和恒等映射机制有效地保留了节点的初始特征，确保即使在深度网络中也能进行有效的信息传输。此外，为了在长文本和短文本上都取得优异的性能，我们将DoubleMix的设计理念应用于XLNet模型，使用混合隐藏状态的混合方法提高了模型的准确性和鲁棒性。实验结果表明，XLG-Net模型在四个基准文本分类数据集上取得了显著的性能提升，验证了该模型在复杂文本分类任务上的有效性。
