Adversarial Scene Editing: Automatic Object Removal from Weak Supervision

2018/06/05 by Rakshith Shetty, Mario Fritz, Shetty, Rakshith +3 · 9 citations
Computer Science · Mathematics · #Advanced Image Processing Techniques #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Digital Media Forensic Detection #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (stat.ML) #cs.AI #cs.CV #stat.ML

paper · pdf · doi:10.48550/arxiv.1806.01911

arxiv created 2018/06/05 · openalex publication_date 2018/06/05 · arxiv updated 2018/06/07 · openalex created_date 2022/09/03 · openalex updated_date 2026/07/28

Abstract

While great progress has been made recently in automatic image manipulation, it has been limited to object centric images like faces or structured scene datasets. In this work, we take a step towards general scene-level image editing by developing an automatic interaction-free object removal model. Our model learns to find and remove objects from general scene images using image-level labels and unpaired data in a generative adversarial network (GAN) framework. We achieve this with two key contributions: a two-stage editor architecture consisting of a mask generator and image in-painter that co-operate to remove objects, and a novel GAN based prior for the mask generator that allows us to flexibly incorporate knowledge about object shapes. We experimentally show on two datasets that our method effectively removes a wide variety of objects using weak supervision only

Cited by

• Privacy-Preserving Computer Vision for Industry: Three Case Studies in Human-Centric Manufacturing
• End-to-End Learning of Geometric Deformations of Feature Maps for Virtual Try-On
• Semantic Image Manipulation Using Scene Graphs
• A Closer Look at the Robustness of Vision-and-Language Pre-trained Models
• MONet: Unsupervised Scene Decomposition and Representation
• Recurrent Feature Reasoning for Image Inpainting
• Semantic Editing On Segmentation Map Via Multi-Expansion Loss
• Enabling Privacy-Aware AI-Based Ergonomic Analysis
• High-Fidelity Image Inpainting with Multimodal Guided GAN Inversion

Related

• EditGAN: High-Precision Semantic Image Editing
• Rethinking the editing of generative adversarial networks: a method to estimate editing vectors based on dimension reduction
• Navigating the GAN Parameter Space for Semantic Image Editing
• SEIGAN: Towards Compositional Image Generation by Simultaneously Learning to Segment, Enhance, and Inpaint
• Decorating Your Own Bedroom: Locally Controlling Image Generation with Generative Adversarial Networks