Be Confident! Towards Trustworthy Graph Neural Networks via Confidence Calibration

2021/09/29 by Xiao Wang, Hongrui Liu, Wang, Xiao +5 · 9 citations
Computer Science · #Advanced Graph Neural Networks #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.AI #cs.LG

paper · pdf · doi:10.48550/arxiv.2109.14285

Accepted to NeurIPS 2021

openalex publication_date 2021/09/29 · openalex created_date 2021/10/11 · arxiv created 2022/01/04 · arxiv updated 2022/01/05 · openalex updated_date 2026/07/28

Abstract

Despite Graph Neural Networks (GNNs) have achieved remarkable accuracy, whether the results are trustworthy is still unexplored. Previous studies suggest that many modern neural networks are over-confident on the predictions, however, surprisingly, we discover that GNNs are primarily in the opposite direction, i.e., GNNs are under-confident. Therefore, the confidence calibration for GNNs is highly desired. In this paper, we propose a novel trustworthy GNN model by designing a topology-aware post-hoc calibration function. Specifically, we first verify that the confidence distribution in a graph has homophily property, and this finding inspires us to design a calibration GNN model (CaGCN) to learn the calibration function. CaGCN is able to obtain a unique transformation from logits of GNNs to the calibrated confidence for each node, meanwhile, such transformation is able to preserve the order between classes, satisfying the accuracy-preserving property. Moreover, we apply the calibration GNN to self-training framework, showing that more trustworthy pseudo labels can be obtained with the calibrated confidence and further improve the performance. Extensive experiments demonstrate the effectiveness of our proposed model in terms of both calibration and accuracy.

Citations

• A New Vector Partition of the Probability Score
• VERIFICATION OF FORECASTS EXPRESSED IN TERMS OF PROBABILITY
• Collective Classification in Network Data
• Semi-Supervised Classification with Graph Convolutional Networks
• Inductive Representation Learning on Large Graphs
• Are Graph Neural Networks Miscalibrated?
• Open Graph Benchmark: Datasets for Machine Learning on Graphs
• Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning
• Trusted Multi-View Classification
• Graph neural networks: A review of methods and applications
• Beyond Low-frequency Information in Graph Convolutional Networks
• A Comprehensive Survey on Graph Neural Networks

Cited by

• HeAD-CP: Heterophily-Aware Diffused Conformal Prediction Sets for Graph Neural Networks
• Two Birds with One Stone: Enhancing Uncertainty Quantification and Interpretability with Graph Functional Neural Process
• Differentiated Information Mining: A Semi-supervised Learning Framework for GNNs
• WATS: Calibrating Graph Neural Networks with Wavelet-Aware Temperature Scaling
• Residual Reweighted Conformal Prediction for Graph Neural Networks
• Uncertainty-Aware Graph Neural Networks: A Multi-Hop Evidence Fusion Approach
• Conf-GNNRec: Quantifying and Calibrating the Prediction Confidence for GNN-based Recommendation Methods
• The Final Layer Holds the Key: A Unified and Efficient GNN Calibration Framework
• Multimodal Graph Representation Learning for Robust Surgical Workflow Recognition with Adversarial Feature Disentanglement

Related

• The Final Layer Holds the Key: A Unified and Efficient GNN Calibration Framework
• What Makes Graph Neural Networks Miscalibrated?
• Are Graph Neural Networks Miscalibrated?
• Confidence-based Graph Convolutional Networks for Semi-Supervised Learning
• Is Your Explanation Reliable: Confidence-Aware Explanation on Graph Neural Networks