TitleMeasuring and Relieving the Over-Smoothing Problem for Graph Neural Networks from the Topological View
AuthorsChen, Deli
Lin, Yankai
Li, Wei
Li, Peng
Zhou, Jie
Sun, Xu
AffiliationPeking Univ, Sch EECS, MOE Key Lab Computat Linguist, Beijing, Peoples R China
Tencent Inc, WeChat AI, Pattern Recognit Ctr, Shenzhen, Peoples R China
Issue Date2020
PublisherTHIRTY-FOURTH AAAI CONFERENCE ON ARTIFICIAL INTELLIGENCE, THE THIRTY-SECOND INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE CONFERENCE AND THE TENTH AAAI SYMPOSIUM ON EDUCATIONAL ADVANCES IN ARTIFICIAL INTELLIGENCE
AbstractGraph Neural Networks (GNNs) have achieved promising performance on a wide range of graph-based tasks. Despite their success, one severe limitation of GNNs is the over-smoothing issue (indistinguishable representations of nodes in different classes). In this work, we present a systematic and quantitative study on the over-smoothing issue of GNNs. First, we introduce two quantitative metrics. MAD and MADGap, to measure the smoothness and oversmoothness of the graph nodes representations, respectively. Then, we verify that smoothing is the nature of GNNs and the critical factor leading to over-smoothness is the low information-to-noise ratio of the message received by the nodes, which is partially determined by the graph topology. Finally, we propose two methods to alleviate the oversmoothing issue from the topological view: (1) MADReg which adds a MADGap-based regularizer to the training objective; (2) AdaEdge which optimizes the graph topology based on the model predictions. Extensive experiments on 7 widely-used graph datasets with 10 typical GNN models show that the two proposed methods are effective for relieving the over-smoothing issue, thus improving the performance of various GNN models.
URIhttp://hdl.handle.net/20.500.11897/618425
ISBN978-1-57735-835-0
ISSN2159-5399
IndexedCPCI-S(ISTP)
Appears in Collections:信息科学技术学院
计算语言学教育部重点实验室

Files in This Work
There are no files associated with this item.

Web of Science®



Checked on Last Week

百度学术™



Checked on Current Time




License: See PKU IR operational policies.