Zhang Qi, Sun Yanfeng, Wang Shaofan, Gao Junbin, Yin Baocai
Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing Institute of Artificial Intelligence, School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China; School of Information Science and Engineering, Shandong Agricultural University, Taian 271018, China.
Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Beijing Institute of Artificial Intelligence, School of Information Science and Technology, Beijing University of Technology, Beijing 100124, China.
Neural Netw. 2025 Nov;191:107792. doi: 10.1016/j.neunet.2025.107792. Epub 2025 Jul 5.
Graph Neural Networks (GNNs) and Graph Transformers (GTs) have shown considerable success in graph-based tasks, each offering distinct strengths: GNNs excel at capturing local details, while GTs are adept at capturing global information. However, both GNNs and GTs face scalability issues when applied to large-scale graphs. To address these challenges, this paper proposes the Graph Transformer Based on Bipartite-stream Information Fusion (BiFormer), a framework designed to integrate the benefits of GTs and GNNs for processing large-scale graphs. BiFormer consists of three modules: a global feature extraction module, which utilizes a Transformer encoder to efficiently capture global information from a small-scale pooled graph; and a local feature extraction module that constructs three parameter-free graph convolution kernels for extracting local features without training; a feature fusion module, which employs a Transformer encoder to fuse extracted local and global features of each node without node-to-node message passing. The complete training of BiFormer requires only the small-scale pooled graph and mini-batched local features to be stored temporarily in memory, allowing for mini-batch training with flexible batch size. Experimental results demonstrate that BiFormer outperforms mainstream GTs and GNNs.
图神经网络（GNN）和图变换器（GT）在基于图的任务中已取得显著成功，各自具有独特优势：GNN擅长捕捉局部细节，而GT则善于捕捉全局信息。然而，GNN和GT在应用于大规模图时都面临可扩展性问题。为应对这些挑战，本文提出了基于二分信息流融合的图变换器（BiFormer），这是一个旨在融合GT和GNN的优势来处理大规模图的框架。BiFormer由三个模块组成：一个全局特征提取模块，它利用变换器编码器从小规模池化图中高效捕捉全局信息；一个局部特征提取模块，它构建三个无参数的图卷积核来提取局部特征而无需训练；一个特征融合模块，它利用变换器编码器融合每个节点提取的局部和全局特征，无需节点到节点的消息传递。BiFormer的完整训练仅需将小规模池化图和小批量局部特征临时存储在内存中，从而允许使用灵活批量大小进行小批量训练。实验结果表明，BiFormer优于主流的GT和GNN。
