School of Information Engineering, Guangdong University of Technology, Guangzhou, China; South China Technology Commercialization Center, Guangzhou, China.
School of Information Engineering, Guangdong University of Technology, Guangzhou, China; Guangdong Provincial Key Laboratory of Intellectual Property and Big Data, Guangzhou, China.
Neural Netw. 2024 Nov;179:106595. doi: 10.1016/j.neunet.2024.106595. Epub 2024 Aug 2.
Graph neural networks (GNNs) leveraging metapaths have garnered extensive utilization. Nevertheless, the escalating parameters and data corpus within graph pre-training models incur mounting training costs. Consequently, GNN models encounter hurdles including diminished generalization capacity and compromised performance amidst small sample datasets. Drawing inspiration from the efficacy demonstrated by self-supervised learning methodologies in natural language processing, we embark on an exploration. We endeavor to imbue graph data with augmentable, learnable prompt vectors targeting node representation enhancement to foster superior adaptability to downstream tasks. This paper proposes a novel approach, the Metapath Integrated Graph Prompt Neural Network (MIGP), which leverages learnable prompt vectors to enhance node representations within a pretrained model framework. By leveraging learnable prompt vectors, MIGP aims to address the limitations posed by mall sample datasets and improve GNNs' model generalization. In the pretraining stage, we split symmetric metapaths in heterogeneous graphs into short metapaths and explicitly propagate information along the metapaths to update node representations. In the prompt-tuning stage, the parameters of the pretrained model are fixed, a set of independent basis vectors is introduced, and an attention mechanism is employed to generate task-specific learnable prompt vectors for each node. Another notable contribution of our work is the introduction of three patent datasets, which is a pioneering application in related fields. We will make these three patent datasets publicly available to facilitate further research on large-scale patent data analysis. Through comprehensive experiments conducted on three patent datasets and three other public datasets, i.e., ACM, IMDB, and DBLP, we demonstrate the superior performance of the MIGP model in enhancing model applicability and performance across a variety of downstream datasets. The source code and datasets are available in the website..
图神经网络 (GNN) 利用元路径得到了广泛的应用。然而，图预训练模型中的参数和数据量不断增加，导致训练成本不断增加。因此，GNN 模型在小样本数据集方面面临着包括概括能力下降和性能下降等挑战。受自然语言处理中自监督学习方法的启发，我们进行了探索。我们试图为图数据赋予可增强的、可学习的提示向量，以增强节点表示，从而提高对下游任务的适应性。本文提出了一种新的方法，即元路径集成图提示神经网络 (MIGP)，它利用可学习的提示向量来增强预训练模型框架中的节点表示。通过利用可学习的提示向量，MIGP 旨在解决小样本数据集带来的限制，并提高 GNN 的模型泛化能力。在预训练阶段，我们将异构图中的对称元路径分割成短元路径，并沿着元路径显式传播信息以更新节点表示。在提示微调阶段，固定预训练模型的参数，引入一组独立的基向量，并使用注意力机制为每个节点生成特定于任务的可学习提示向量。我们工作的另一个贡献是引入了三个专利数据集，这是相关领域的开创性应用。我们将公开这三个专利数据集，以促进对大规模专利数据分析的进一步研究。通过在三个专利数据集和三个其他公共数据集（即 ACM、IMDB 和 DBLP）上进行全面实验，我们证明了 MIGP 模型在增强模型适用性和性能方面的优越性能，适用于各种下游数据集。代码和数据集可在网站上获取。
