Invariant Representations through Adversarial Forgetting

2019/11/11 by Jaiswal, Ayush, Moyer, Daniel, Steeg, Greg Ver +2 · 2 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.1911.04060

Abstract

We propose a novel approach to achieving invariance for deep neural networks in the form of inducing amnesia to unwanted factors of data through a new adversarial forgetting mechanism. We show that the forgetting mechanism serves as an information-bottleneck, which is manipulated by the adversarial training to learn invariance to unwanted factors. Empirical results show that the proposed framework achieves state-of-the-art performance at learning invariance in both nuisance and bias settings on a diverse collection of datasets and tasks.

Cited by

• Fair Representation Learning with Controllable High Confidence Guarantees via Adversarial Inference
• FairContrast: Enhancing Fairness through Contrastive learning and Customized Augmenting Methods on Tabular Data

Related

• Unified Adversarial Invariance
• Unsupervised Adversarial Invariance
• The Effectiveness of Random Forgetting for Robust Generalization
• On the Implicit Adversariality of Catastrophic Forgetting in Deep Continual Learning
• Invariant Causal Mechanisms through Distribution Matching