Learning Less-Overlapping Representations

2017/11/25 by Pengtao Xie, Hongbao Zhang, Xie, Pengtao +3
Computer Science · #Domain Adaptation and Few-Shot Learning #Human Pose and Action Recognition #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.1711.09300

Abstract

In representation learning (RL), how to make the learned representations easy to interpret and less overfitted to training data are two important but challenging issues. To address these problems, we study a new type of regulariza- tion approach that encourages the supports of weight vectors in RL models to have small overlap, by simultaneously promoting near-orthogonality among vectors and sparsity of each vector. We apply the proposed regularizer to two models: neural networks (NNs) and sparse coding (SC), and develop an efficient ADMM-based algorithm for regu- larized SC. Experiments on various datasets demonstrate that weight vectors learned under our regularizer are more interpretable and have better generalization performance.

Citations

• The Mythos of Model Interpretability

Related

• \boldsymbolλ-Orthogonality Regularization for Compatible Representation Learning
• Only sparsity based loss function for learning representations
• Feature Incay for Representation Regularization
• Hierarchical regularization networks for sparsification based learning on noisy datasets
• Learning Sparse Latent Representations for Generator Model