Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks

2016/12/16 by Konstantinos Bousmalis, Nathan Silberman, Bousmalis, Konstantinos +8 · 20 citations
Computer Science · Mathematics · #Adversarial system #Artificial intelligence #Classifier (UML) #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Domain (mathematical analysis) #Domain Adaptation and Few-Shot Learning #Domain adaptation #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Generative grammar #Image (mathematics) #Machine learning #Mathematics #Multimodal Machine Learning Applications #Pattern recognition (psychology) #Rendering (computer graphics) #Transformation (genetics) #Unsupervised learning #cs.CV

paper · pdf · doi:10.48550/arxiv.1612.05424

published in arXiv (Cornell University) (Cornell University) · Final CVPR 2017 paper and supplementary material

openalex publication_date 2016/12/16 · arxiv created 2017/08/23 · arxiv updated 2017/08/24 · openalex created_date 2022/10/05 · openalex updated_date 2026/08/05

Abstract

Collecting well-annotated image datasets to train modern machine learning algorithms is prohibitively expensive for many tasks. One appealing alternative is rendering synthetic data where ground-truth annotations are generated automatically. Unfortunately, models trained purely on rendered images often fail to generalize to real images. To address this shortcoming, prior work introduced unsupervised domain adaptation algorithms that attempt to map representations between the two domains or learn to extract features that are domain-invariant. In this work, we present a new approach that learns, in an unsupervised manner, a transformation in the pixel space from one domain to the other. Our generative adversarial network (GAN)-based method adapts source-domain images to appear as if drawn from the target domain. Our approach not only produces plausible samples, but also outperforms the state-of-the-art on a number of unsupervised domain adaptation scenarios by large margins. Finally, we demonstrate that the adaptation process generalizes to object classes unseen during training.

Citations

• Adam: A Method for Stochastic Optimization
• Deep Domain Confusion: Maximizing for Domain Invariance
• Domain-Adversarial Neural Networks
• Gradient-based learning applied to document recognition
• Learning Transferable Features with Deep Adaptation Networks
• Coupled Generative Adversarial Networks
• UnrealCV: Connecting Computer Vision to Unreal Engine
• Target-driven Visual Navigation in Indoor Scenes using Deep Reinforcement Learning
• ResearchDoom and CocoDoom: Learning Computer Vision with Games
• Perceptual Losses for Real-Time Style Transfer and Super-Resolution

Cited by

• Saliency-Guided Domain Adaptation for Left-Hand Driving in Autonomous Steering
• High-Fidelity Simulated Data Generation for Real-World Zero-Shot Robotic Manipulation Learning with Gaussian Splatting
• Signal Classification Recovery Across Domains Using Unsupervised Domain Adaptation
• Generalizable Domain Adaptation for Sim-and-Real Policy Co-Training
• Domain Adaptation via Feature Refinement
• IntrinsicReal: Adapting IntrinsicAnything from Synthetic to Real Objects
• How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy
• CORE-ReID V2: Advancing the Domain Adaptation for Object Re-Identification with Optimized Training and Ensemble Fusion
• Rein++: Efficient Generalization and Adaptation for Semantic Segmentation with Vision Foundation Models
• Stable-Sim2Real: Exploring Simulation of Real-Captured 3D Data with Two-Stage Depth Diffusion
• Cross-Resolution SAR Target Detection Using Structural Hierarchy Adaptation and Reliable Adjacency Alignment
• Model Compression using Progressive Channel Pruning
• RoboPearls: Editable Video Simulation for Robot Manipulation
• Certainty and Uncertainty Guided Active Domain Adaptation
• Continuous Domain Generalization
• Bridging the Reality Gap in Digital Twins with Context-Aware, Physics-Guided Deep Learning
• Stabilizing In-Context Multi-Source Domain Adaptation for Biomedical Images Through Controls
• Identity Preserving Generative Adversarial Network for Cross-Domain Person Re-identification
• Monitor and Recover: A Paradigm for Future Research on Distribution Shift in Learning-Enabled Cyber-Physical Systems
• CRAFT: Complementary Recommendations Using Adversarial Feature Transformer

Related

• Generate To Adapt: Aligning Domains using Generative Adversarial Networks
• Adversarial Feature Augmentation for Unsupervised Domain Adaptation
• Preserving Semantic Consistency in Unsupervised Domain Adaptation Using Generative Adversarial Networks
• Domain-Invariant Adversarial Learning for Unsupervised Domain Adaption
• Transferring GANs: generating images from limited data