Unsupervised Discovery of Interpretable Directions in the GAN Latent Space

2020/02/10 by Andrey Voynov, Voynov, Andrey, Artem Babenko +1 · 142 citations
Computer Science · Mathematics · #Artificial intelligence #Computer Vision and Pattern Recognition (cs.CV) #Computer science #FOS: Computer and information sciences #Image Processing and 3D Reconstruction #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques #Space (punctuation) #Speech Recognition and Synthesis #cs.CV #cs.LG #stat.ML

paper · pdf · doi:10.48550/arxiv.2002.03754

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2020/02/10 · arxiv created 2020/06/24 · arxiv updated 2020/06/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

The latent spaces of GAN models often have semantically meaningful directions. Moving in these directions corresponds to human-interpretable image transformations, such as zooming or recoloring, enabling a more controllable generation process. However, the discovery of such directions is currently performed in a supervised manner, requiring human labels, pretrained models, or some form of self-supervision. These requirements severely restrict a range of directions existing approaches can discover. In this paper, we introduce an unsupervised method to identify interpretable directions in the latent space of a pretrained GAN model. By a simple model-agnostic procedure, we find directions corresponding to sensible semantic manipulations without any form of (self-)supervision. Furthermore, we reveal several non-trivial findings, which would be difficult to obtain by existing methods, e.g., a direction corresponding to background removal. As an immediate practical benefit of our work, we show how to exploit this finding to achieve competitive performance for weakly-supervised saliency detection.

Citations

• Gradient-based learning applied to document recognition
• Video-to-Video Synthesis
• Large Scale GAN Training for High Fidelity Natural Image Synthesis
• A Spectral Regularizer for Unsupervised Disentanglement
• InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets
• Interpreting the Latent Space of GANs for Semantic Face Editing
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
• Controlling generative models with continuous factors of variations

Cited by

• Learning Common and Salient Generative Factors Between Two Image Datasets
• Beyond the Black Box: Identifiable Interpretation and Control in Generative Models via Causal Minimality
• Which Way from B to A: The role of embedding geometry in image interpolation for Stable Diffusion
• SCALEX: Scalable Concept and Latent Exploration for Diffusion Models
• Disentangled Representation Learning via Modular Compositional Bias
• Knowledge Abstraction for Knowledge-based Semantic Communication: A Generative Causality Invariant Approach
• Knowledge-integrated AutoEncoder Model
• EnCoBo: Energy-Guided Concept Bottlenecks for Interpretable Generation
• Unfolding Generative Flows with Koopman Operators: Fast and Interpretable Sampling
• From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit
• Ensembling with Deep Generative Views
• Enhancing Interpretability of Sparse Latent Representations with Class Information
• XFACTORS: Disentangled Information Bottleneck via Contrastive Supervision
• Image Interpolation with Score-based Riemannian Metrics of Diffusion Models
• CA-GAN: Weakly Supervised Color Aware GAN for Controllable Makeup Transfer

Related

• Unsupervised Discovery of Disentangled Manifolds in GANs
• LatentCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions
• GAN "Steerability" without optimization
• Discovering Class-Specific GAN Controls for Semantic Image Synthesis
• Fantastic Style Channels and Where to Find Them: A Submodular Framework for Discovering Diverse Directions in GANs