Generalizing Across Domains via Cross-Gradient Training

2018/04/28 by Shiv Shankar, Shankar, Shiv, Vihari Piratla +9 · 26 citations
Computer Science · Mathematics · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Speech Recognition and Synthesis #Topic Modeling #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1804.10745

The first two authors contributed equally; Accepted at ICLR 2018

openalex publication_date 2018/04/28 · arxiv created 2018/05/01 · arxiv updated 2018/05/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present CROSSGRAD, a method to use multi-domain training data to learn a classifier that generalizes to new domains. CROSSGRAD does not need an adaptation phase via labeled or unlabeled data, or domain features in the new domain. Most existing domain adaptation methods attempt to erase domain signals using techniques like domain adversarial training. In contrast, CROSSGRAD is free to use domain signals for predicting labels, if it can prevent overfitting on training domains. We conceptualize the task in a Bayesian setting, in which a sampling step is implemented as data augmentation, based on domain-guided perturbations of input instances. CROSSGRAD parallelly trains a label and a domain classifier on examples perturbed by loss gradients of each other's objectives. This enables us to directly perturb inputs, without separating and re-mixing domain signals while making various distributional assumptions. Empirical evaluation on three different applications where this setting is natural establishes that (1) domain-guided perturbation provides consistently better generalization to unseen domains, compared to generic instance perturbation methods, and that (2) data augmentation is a more stable and accurate method than domain adversarial training.

Citations

• Intriguing properties of neural networks
• Gradient-based learning applied to document recognition
• Adversarial Discriminative Domain Adaptation

Cited by

• Multimodal Domain Generalization for Depression Detection: An Attention-Based BiLSTM Network With Domain-Adversarial Training
• ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction
• Environment Inference for Learning Generalizable Dynamical System
• SDGraph: Multi-Level Sketch Representation Learning by Sparse-Dense Graph Architecture
• Variational Disentanglement for Domain Generalization
• ConstStyle: Robust Domain Generalization with Unified Style Transformation
• Adapt in the Wild: Test-Time Entropy Minimization with Sharpness and Feature Regularization
• Single Domain Generalization in Diabetic Retinopathy: A Neuro-Symbolic Learning Approach
• In Search of Lost Domain Generalization
• Class Unbiasing for Generalization in Medical Diagnosis
• PatchDSU: Uncertainty Modeling for Out of Distribution Generalization in Keyword Spotting
• Representation via Representations: Domain Generalization via Adversarially Learned Invariant Representations
• Zero-shot Domain Adaptation without Domain Semantic Descriptors
• Simulate, Refocus and Ensemble: An Attention-Refocusing Scheme for Domain Generalization
• Domain Generalization via Universal Non-volume Preserving Models
• FEDTAIL: Federated Long-Tailed Domain Generalization with Sharpness-Guided Gradient Matching
• Rule Learning for Knowledge Graph Reasoning under Agnostic Distribution Shift
• FixCLR: Negative-Class Contrastive Learning for Semi-Supervised Domain Generalization
• Deep Stable Learning for Out-Of-Distribution Generalization
• Discovering Spatial Relationships by Transformers for Domain Generalization
• Untapped Potential of Data Augmentation: A Domain Generalization Viewpoint
• Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference
• A Bit More Bayesian: Domain-Invariant Learning with Uncertainty
• Feature Alignment and Restoration for Domain Generalization and Adaptation
• CLIP-Powered Domain Generalization and Domain Adaptation: A Comprehensive Survey
• Domain Generalization via Semi-supervised Meta Learning
• Self-Bootstrapping for Versatile Test-Time Adaptation

Related

• Domain Generalization via Gradient Surgery
• CADG: A Model Based on Cross Attention for Domain Generalization
• A Novel Cross-Perturbation for Single Domain Generalization
• Cross-Domain Ensemble Distillation for Domain Generalization
• Learning to Generalize: Meta-Learning for Domain Generalization