Enforcing Predictive Invariance across Structured Biomedical Domains

2020/06/06 by Wengong Jin, Regina Barzilay, Jin, Wengong +3 · 2 citations
Computer Science · Materials Science · #Computational Drug Discovery Methods #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Materials Science #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2006.03908

openalex publication_date 2020/06/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Many biochemical applications such as molecular property prediction require models to generalize beyond their training domains (environments). Moreover, natural environments in these tasks are structured, defined by complex descriptors such as molecular scaffolds or protein families. Therefore, most environments are either never seen during training, or contain only a single training example. To address these challenges, we propose a new regret minimization (RGM) algorithm and its extension for structured environments. RGM builds from invariant risk minimization (IRM) by recasting simultaneous optimality condition in terms of predictive regret, finding a representation that enables the predictor to compete against an oracle with hindsight access to held-out environments. The structured extension adaptively highlights variation due to complex environments via specialized domain perturbations. We evaluate our method on multiple applications: molecular property prediction, protein homology and stability prediction and show that RGM significantly outperforms previous state-of-the-art baselines.

Citations

• Domain Generalization via Invariant Feature Representation
• Discriminative Embeddings of Latent Variable Models for Structured Data
• Conditional Adversarial Domain Adaptation
• On Learning Invariant Representation for Domain Adaptation
• Invariant Risk Minimization
• Domain Generalization via Model-Agnostic Learning of Semantic Features
• Out-of-Distribution Generalization via Risk Extrapolation (REx)

Cited by

• Learning Stable Classifiers by Transferring Unstable Features
• Towards a Theoretical Framework of Out-of-Distribution Generalization

Related

• Adaptive Invariance for Molecule Property Prediction
• A Pipeline for Data-Driven Learning of Topological Features with Applications to Protein Stability Prediction
• ReLSO: A Transformer-based Model for Latent Space Optimization and Generation of Proteins
• Robust Molecular Property Prediction via Densifying Scarce Labeled Data
• Fighting against uncertainty: An essential issue in bioinformatics