Self-Supervised Learning on Molecular Graphs: A Systematic Investigation of Masking Design

2025/12/08 by Yang, Jiannan, Thost, Veronika, Ma, Tengfei · 1 citation
Computer Science · Materials Science · #Advanced Graph Neural Networks #Artificial Intelligence (cs.AI) #Computational Drug Discovery Methods #Encoder #FOS: Biological sciences #FOS: Computer and information sciences #Graph #Heuristics #Machine Learning (cs.LG) #Machine Learning in Materials Science #Masking (illustration) #Probabilistic logic #Quantitative Methods (q-bio.QM) #Transformer #Workflow

paper · open access · doi:10.48550/arxiv.2512.07064

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/12/08 · openalex created_date 2025/12/10 · openalex updated_date 2026/07/28

Abstract

Self-supervised learning (SSL) plays a central role in molecular representation learning. Yet, many recent innovations in masking-based pretraining are introduced as heuristics and lack principled evaluation, obscuring which design choices are genuinely effective. This work cast the entire pretrain-finetune workflow into a unified probabilistic framework, enabling a transparent comparison and deeper understanding of masking strategies. Building on this formalism, we conduct a controlled study of three core design dimensions: masking distribution, prediction target, and encoder architecture, under rigorously controlled settings. We further employ information-theoretic measures to assess the informativeness of pretraining signals and connect them to empirically benchmarked downstream performance. Our findings reveal a surprising insight: sophisticated masking distributions offer no consistent benefit over uniform sampling for common node-level prediction tasks. Instead, the choice of prediction target and its synergy with the encoder architecture are far more critical. Specifically, shifting to semantically richer targets yields substantial downstream improvements, particularly when paired with expressive Graph Transformer encoders. These insights offer practical guidance for developing more effective SSL methods for molecular graphs.

Citations

• SMI-Editor: Edit-based SMILES Language Model with Fragment-level Supervision
• MaskMol: Knowledge-guided Molecular Image Pre-Training Framework for Activity Cliffs
• Where to Mask: Structure-Guided Masking for Graph Masked Autoencoders
• An end-to-end attention-based approach for learning on graphs
• UGMAE: A Unified Framework for Graph Masked Autoencoders
• Generative and Contrastive Paradigms Are Complementary for Graph Self-Supervised Learning
• Rethinking Tokenizer and Decoder in Masked Graph Modeling for Molecules
• Motif-aware Attribute Masking for Molecular Graph Pre-training
• Evaluating Self-Supervised Learning for Molecular Graph Embeddings
• Recipe for a General, Powerful, Scalable Graph Transformer
• GraphMAE: Self-Supervised Masked Graph Autoencoders
• Masked Autoencoders Are Scalable Vision Learners
• Pre-training Molecular Graph Representation with 3D Geometry
• Motif-based Graph Self-Supervised Learning for Molecular Property Prediction
• Graph Contrastive Learning with Augmentations
• Strategies for Pre-training Graph Neural Networks
• Self-supervised Visual Feature Learning with Deep Neural Networks: A Survey
• How Powerful are Graph Neural Networks?
• Neural Discrete Representation Learning
• Neural Message Passing for Quantum Chemistry
• MoleculeNet: A Benchmark for Molecular Machine Learning
• Convolutional Networks on Graphs for Learning Molecular Fingerprints
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• A Comprehensive Survey on Graph Neural Networks

Cited by

• Oracle-Budgeted Molecular Optimization with Short-Term Graph Memory

Related

• Rethinking Tokenizer and Decoder in Masked Graph Modeling for Molecules
• Attention-wise masked graph contrastive learning for predicting molecular property
• Self-Supervised Graph Transformer on Large-Scale Molecular Data
• Does GNN Pretraining Help Molecular Representation?
• MolCPT: Molecule Continuous Prompt Tuning to Generalize Molecular Representation Learning