Discovery and Separation of Features for Invariant Representation Learning

2019/12/02 by Ayush Jaiswal, Rob Brekelmans, Jaiswal, Ayush +10 · 1 citation
Computer Science · Mathematics · #Anomaly Detection Techniques and Applications #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.1912.00646

10 pages, 3 figures

arxiv created 2019/12/02 · arxiv updated 2019/12/03

Abstract

Supervised machine learning models often associate irrelevant nuisance factors with the prediction target, which hurts generalization. We propose a framework for training robust neural networks that induces invariance to nuisances through learning to discover and separate predictive and nuisance factors of data. We present an information theoretic formulation of our approach, from which we derive training objectives and its connections with previous methods. Empirical results on a wide array of datasets show that the proposed framework achieves state-of-the-art performance, without requiring nuisance annotations during training.

Citations

• Gradient-based learning applied to document recognition
• ImageNet classification with deep convolutional neural networks

Cited by

• Robust Classification under Class-Dependent Domain Shift

Related

• Unified Adversarial Invariance
• Unsupervised Adversarial Invariance
• Invariant Representations with Stochastically Quantized Neural Networks
• Learning Invariant Representations with a Nonparametric Nadaraya-Watson Head
• Invariant Representations without Adversarial Training