Li Haifeng, Cao Jun, Zhu Jiawei, Luo Qinyao, He Silu, Wang Xuying
IEEE Trans Neural Netw Learn Syst. 2024 Aug;35(8):11157-11167. doi: 10.1109/TNNLS.2023.3248871. Epub 2024 Aug 5.
Graph contrastive learning (GCL) is a promising direction toward alleviating the label dependence, poor generalization and weak robustness of graph neural networks, learning representations with invariance, and discriminability by solving pretasks. The pretasks are mainly built on mutual information estimation, which requires data augmentation to construct positive samples with similar semantics to learn invariant signals and negative samples with dissimilar semantics to empower representation discriminability. However, an appropriate data augmentation configuration depends heavily on lots of empirical trials such as choosing the compositions of data augmentation techniques and the corresponding hyperparameter settings. We propose an augmentation-free GCL method, invariant-discriminative GCL (iGCL), that does not intrinsically require negative samples. iGCL designs the invariant-discriminative loss (ID loss) to learn invariant and discriminative representations. On the one hand, ID loss learns invariant signals by directly minimizing the mean square error (MSE) between the target samples and positive samples in the representation space. On the other hand, ID loss ensures that the representations are discriminative by an orthonormal constraint forcing the different dimensions of representations to be independent of each other. This prevents representations from collapsing to a point or subspace. Our theoretical analysis explains the effectiveness of ID loss from the perspectives of the redundancy reduction criterion, canonical correlation analysis (CCA), and information bottleneck (IB) principle. The experimental results demonstrate that iGCL outperforms all baselines on five node classification benchmark datasets. iGCL also shows superior performance for different label ratios and is capable of resisting graph attacks, which indicates that iGCL has excellent generalization and robustness. The source code is available at https://github.com/lehaifeng/ T-GCN/tree/master/iGCL.
图对比学习（GCL）是减轻图神经网络标签依赖、泛化能力差和鲁棒性弱的一个有前景的方向，通过解决预任务来学习具有不变性和可区分性的表示。预任务主要基于互信息估计构建，这需要数据增强来构造具有相似语义的正样本以学习不变信号，以及具有不同语义的负样本以增强表示的可区分性。然而，合适的数据增强配置在很大程度上依赖于大量的经验试验，例如选择数据增强技术的组合以及相应的超参数设置。我们提出一种无需增强的GCL方法，即不变-可区分GCL（iGCL），它本质上不需要负样本。iGCL设计了不变-可区分损失（ID损失）来学习不变和可区分的表示。一方面，ID损失通过直接最小化表示空间中目标样本和正样本之间的均方误差（MSE）来学习不变信号。另一方面，ID损失通过正交约束确保表示具有可区分性，该约束迫使表示的不同维度相互独立。这防止了表示坍缩到一个点或子空间。我们的理论分析从冗余减少准则、典型相关分析（CCA）和信息瓶颈（IB）原理的角度解释了ID损失的有效性。实验结果表明，在五个节点分类基准数据集上，iGCL优于所有基线。iGCL在不同标签比例下也表现出卓越的性能，并且能够抵御图攻击，这表明iGCL具有出色的泛化能力和鲁棒性。源代码可在https://github.com/lehaifeng/T-GCN/tree/master/iGCL获取。
