MaskGAN: Towards Diverse and Interactive Facial Image Manipulation

2019/07/27 by Cheng‐Han Lee, Lee, Cheng-Han, Ziwei Liu +5 · 33 citations
Computer Science · Medicine · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Face recognition and analysis #Facial Nerve Paralysis Treatment and Research #Generative Adversarial Networks and Image Synthesis #Graphics (cs.GR) #Machine Learning (cs.LG)

paper · pdf · doi:10.48550/arxiv.1907.11922

openalex publication_date 2019/07/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Facial image manipulation has achieved great progress in recent years. However, previous methods either operate on a predefined set of face attributes or leave users little freedom to interactively manipulate images. To overcome these drawbacks, we propose a novel framework termed MaskGAN, enabling diverse and interactive face manipulation. Our key insight is that semantic masks serve as a suitable intermediate representation for flexible face manipulation with fidelity preservation. MaskGAN has two main components: 1) Dense Mapping Network (DMN) and 2) Editing Behavior Simulated Training (EBST). Specifically, DMN learns style mapping between a free-form user modified mask and a target image, enabling diverse generation results. EBST models the user editing behavior on the source mask, making the overall framework more robust to various manipulated inputs. Specifically, it introduces dual-editing consistency as the auxiliary supervision signal. To facilitate extensive studies, we construct a large-scale high-resolution face dataset with fine-grained mask annotations named CelebAMask-HQ. MaskGAN is comprehensively evaluated on two challenging tasks: attribute transfer and style copy, demonstrating superior performance over other state-of-the-art methods. The code, models, and dataset are available at https://github.com/switchablenorms/CelebAMask-HQ.

Citations

• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Instance Normalization: The Missing Ingredient for Fast Stylization

Cited by

• PixelArena: A benchmark for Pixel-Precision Visual Intelligence
• Improving the Plausibility of Pressure Distributions Synthesized from Depth Image through Generative Modeling
• Open-World Deepfake Attribution via Confidence-Aware Asymmetric Learning
• TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detection
• Fully Unsupervised Self-debiasing of Text-to-Image Diffusion Models
• Training-Free Generation of Diverse and High-Fidelity Images via Prompt Semantic Space Optimization
• Multivariate Diffusion Transformer with Decoupled Attention for High-Fidelity Mask-Text Collaborative Facial Generation
• DiffSwap++: 3D Latent-Controlled Diffusion for Identity-Preserving Face Swapping
• Deep Generative Models for Enhanced Vitreous OCT Imaging
• Face De-Identification: A Domain-Centric Survey from Capture to Processing
• TIGA: Trajectory-Injected Generative Attack against Black-box AIGC Detectors
• Med-SegLens: Latent-Level Model Diffing for Interpretable Medical Image Segmentation
• Prototype-Grounded Concept Models for Verifiable Concept Alignment
• Privacy-Preserving Semantic Communication over Wiretap Channels with Learnable Differential Privacy
• Fake-in-Facext: Towards Fine-Grained Explainable DeepFake Analysis
• Efficient Few-shot Identity Preserving Attribute Editing for 3D-aware Deep Generative Models
• Efficient Label Refinement for Face Parsing Under Extreme Poses Using 3D Gaussian Splatting
• Bayesian Neural Networks for Functional ANOVA model
• BFSM: 3D Bidirectional Face-Skull Morphable Model
• Universal Multi-Domain Translation via Diffusion Routers
• PerFace: Metric Learning in Perceptual Facial Similarity for Enhanced Face Anonymization
• TongueReenact: Geometry-Anchored Tongue Synthesis for Face Reenactment
• SISMA: Semantic Face Image Synthesis with Mamba
• Bias Amplification in Stable Diffusion's Representation of Stigma Through Skin Tones and Their Homogeneity
• Face4FairShifts: A Large Image Benchmark for Fairness and Robust Learning across Visual Domains
• Mixture of Global and Local Experts with Diffusion Transformer for Controllable Face Generation
• Enhancing Diffusion Face Generation with Contrastive Embeddings and SegFormer Guidance
• PaCo-FR: Patch-Pixel Aligned End-to-End Codebook Learning for Facial Representation Pre-training
• Never compromise with vulnerabilities: a comprehensive survey on AI governance
• CycleDiff: Cycle Diffusion Models for Unpaired Image-to-image Translation
• How Does Bilateral Ear Symmetry Affect Deep Ear Features?
• Uncertainty-Guided Face Matting for Occlusion-Aware Face Transformation
• DisFaceRep: Representation Disentanglement for Co-occurring Facial Components in Weakly Supervised Face Parsing

Related

• FacialGAN: Style Transfer and Attribute Manipulation on Synthetic Faces
• MagGAN: High-Resolution Face Attribute Editing with Mask-Guided Generative Adversarial Network
• Mask-Guided Portrait Editing with Conditional GANs
• High Resolution Face Editing with Masked GAN Latent Code Optimization
• S2FGAN: Semantically Aware Interactive Sketch-to-Face Translation