GraphMAE: Self-Supervised Masked Graph Autoencoders

2022/05/22 by Zhenyu Hou, Hou, Zhenyu, Xiao Liu +11 · 46 citations
Computer Science · #Advanced Graph Neural Networks #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.2205.10803

openalex publication_date 2022/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Self-supervised learning (SSL) has been extensively explored in recent years. Particularly, generative SSL has seen emerging success in natural language processing and other AI fields, such as the wide adoption of BERT and GPT. Despite this, contrastive learning-which heavily relies on structural data augmentation and complicated training strategies-has been the dominant approach in graph SSL, while the progress of generative SSL on graphs, especially graph autoencoders (GAEs), has thus far not reached the potential as promised in other fields. In this paper, we identify and examine the issues that negatively impact the development of GAEs, including their reconstruction objective, training robustness, and error metric. We present a masked graph autoencoder GraphMAE that mitigates these issues for generative self-supervised graph pretraining. Instead of reconstructing graph structures, we propose to focus on feature reconstruction with both a masking strategy and scaled cosine error that benefit the robust training of GraphMAE. We conduct extensive experiments on 21 public datasets for three different graph learning tasks. The results manifest that GraphMAE-a simple graph autoencoder with careful designs-can consistently generate outperformance over both contrastive and generative state-of-the-art baselines. This study provides an understanding of graph autoencoders and demonstrates the potential of generative self-supervised pre-training on graphs.

Cited by

• Unsupervised Graph Representation Learning with Complementary View Alignment
• From Pixels to Predicates Structuring urban perception with scene graphs
• Trustworthy Neighborhoods Mining: Homophily-Aware Neutral Contrastive Learning for Graph Clustering
• Feature-Centric Unsupervised Node Representation Learning Without Homophily Assumption
• CORE: Contrastive Masked Feature Reconstruction on Graphs
• Attacking and Securing Community Detection: A Game-Theoretic Framework
• Self-Supervised Learning on Molecular Graphs: A Systematic Investigation of Masking Design
• Mitigating Structural Overfitting: A Distribution-Aware Rectification Framework for Missing Feature Imputation
• FGC-Comp: Adaptive Neighbor-Grouped Attribute Completion for Graph-based Anomaly Detection
• UniCom: Towards a Unified and Cohesiveness-aware Framework for Community Search and Detection
• MolSight: Optical Chemical Structure Recognition with SMILES Pretraining, Multi-Granularity Learning and Reinforcement Learning
• One Model for All: Universal Pre-training for EEG based Emotion Recognition across Heterogeneous Datasets and Paradigms
• Versatile and Risk-Sensitive Cardiac Diagnosis via Graph-Based ECG Signal Representation
• KGFR: A Foundation Retriever for Generalized Knowledge Graph Question Answering
• DeNoise: Learning Robust Graph Representations for Unsupervised Graph-Level Anomaly Detection
• GRAVER: Generative Graph Vocabularies for Robust Graph Foundation Models Fine-tuning
• Equilibrium Policy Generalization: A Reinforcement Learning Framework for Cross-Graph Zero-Shot Generalization in Pursuit-Evasion Games
• Empowering LLMs with Structural Role Inference for Zero-Shot Graph Learning
• AgentGFM: A Graph Foundation Model with Node-Agent Information-Flow Control
• SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning
• GraphTOP: Graph Topology-Oriented Prompting for Graph Neural Networks
• Leveraging Classical Algorithms for Graph Neural Networks
• From Masks to Worlds: A Hitchhiker's Guide to World Models
• Learning Noise-Resilient and Transferable Graph-Text Alignment via Dynamic Quality Assessment
• 3D-GSRD: 3D Molecular Graph Auto-Encoder with Selective Re-mask Decoding
• UniGTE: Unified Graph-Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains
• A Hierarchical Quantized Tokenization Framework for Task-Adaptive Graph Representation Learning
• Higher-order interactions of multi-layer prompt
• Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretraining
• One Prompt Fits All: Universal Graph Adaptation for Pretrained Models
• SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs
• Same Graph Cross-Task Transfer in GNNs: Protocols and Predictors
• SpatialFusion: A lightweight multimodal foundation model for pathway-informed spatial niche mapping
• Self-Supervised Learning of Graph Representations for Network Intrusion Detection
• BIGNet: Pretrained Graph Neural Network for Embedding Semantic, Spatial, and Topological Data in BIM Models
• LMAE4Eth: Generalizable and Robust Ethereum Fraud Detection by Exploring Transaction Semantics and Masked Graph Embedding
• GSTBench: A Benchmark Study on the Transferability of Graph Self-Supervised Learning
• LLM-based Contrastive Self-Supervised AMR Learning with Masked Graph Autoencoders for Fake News Detection
• Beyond Tokens: Enhancing RTL Quality Estimation via Structural Graph Learning
• Aligning the Spectrum: Hybrid Graph Pre-training and Prompt Tuning across Homophily and Heterophily
• GRAVITY: A Controversial Graph Representation Learning for Vertex Classification
• Towards Real-World Rumor Detection: Anomaly Detection Framework with Graph Supervised Contrastive Learning
• Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learning
• Hypergraph Neural Network with State Space Models for Node Classification
• GFlowNets for Learning Better Drug-Drug Interaction Representations
• Graph Representation Learning with Massive Unlabeled Data for Rumor Detection

Related

• GraphMAE2: A Decoding-Enhanced Masked Self-Supervised Graph Learner
• Rethinking Graph Masked Autoencoders through Alignment and Uniformity
• What's Behind the Mask: Understanding Masked Graph Modeling for Graph Autoencoders
• Heterogeneous Graph Masked Autoencoders
• Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning