Li Jin, Zhang Qirong, Liu Wenxi, Chan Antoni B, Fu Yang-Geng
IEEE Trans Neural Netw Learn Syst. 2025 Apr;36(4):6897-6910. doi: 10.1109/TNNLS.2024.3402317. Epub 2025 Apr 4.
Graph neural networks (GNNs) are widely used for analyzing graph-structural data and solving graph-related tasks due to their powerful expressiveness. However, existing off-the-shelf GNN-based models usually consist of no more than three layers. Deeper GNNs usually suffer from severe performance degradation due to several issues including the infamous "over-smoothing" issue, which restricts the further development of GNNs. In this article, we investigate the over-smoothing issue in deep GNNs. We discover that over-smoothing not only results in indistinguishable embeddings of graph nodes, but also alters and even corrupts their semantic structures, dubbed semantic over-smoothing. Existing techniques, e.g., graph normalization, aim at handling the former concern, but neglect the importance of preserving the semantic structures in the spatial domain, which hinders the further improvement of model performance. To alleviate the concern, we propose a cluster-keeping sparse aggregation strategy to preserve the semantic structure of embeddings in deep GNNs (especially for spatial GNNs). Particularly, our strategy heuristically redistributes the extent of aggregations for all the nodes from layers, instead of aggregating them equally, so that it enables aggregate concise yet meaningful information for deep layers. Without any bells and whistles, it can be easily implemented as a plug-and-play structure of GNNs via weighted residual connections. Last, we analyze the over-smoothing issue on the GNNs with weighted residual structures and conduct experiments to demonstrate the performance comparable to the state-of-the-arts.
图神经网络（GNN）因其强大的表达能力而被广泛用于分析图结构数据和解决与图相关的任务。然而，现有的基于现成GNN的模型通常由不超过三层组成。更深层次的GNN通常会由于包括臭名昭著的“过平滑”问题在内的几个问题而遭受严重的性能下降，这限制了GNN的进一步发展。在本文中，我们研究了深度GNN中的过平滑问题。我们发现过平滑不仅会导致图节点的嵌入无法区分，还会改变甚至破坏它们的语义结构，即语义过平滑。现有技术，如图归一化，旨在处理前一个问题，但忽略了在空间域中保留语义结构的重要性，这阻碍了模型性能的进一步提高。为了缓解这一问题，我们提出了一种保持聚类的稀疏聚合策略，以保留深度GNN（特别是空间GNN）中嵌入的语义结构。特别是，我们的策略启发式地重新分配了来自各层所有节点的聚合程度，而不是平均聚合它们，从而使深度层能够聚合简洁而有意义的信息。无需任何花里胡哨的东西，它可以通过加权残差连接轻松实现为GNN的即插即用结构。最后，我们分析了具有加权残差结构的GNN上的过平滑问题，并进行实验以证明其性能与最先进技术相当。
