2021/03/18 by Lucy Chai, Chai, Lucy, Jonas Wulff +3 · 1 citation
Computer Science · #Advanced Image Processing Techniques #Computer Vision and Pattern Recognition (cs.CV) #Digital Media Forensic Detection #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning in Healthcare #Topic Modeling
paper · pdf · doi:10.48550/arxiv.2103.10426
openalex publication_date 2021/03/18 · openalex created_date 2021/03/29 · openalex updated_date 2026/07/28
In recent years, Generative Adversarial Networks have become ubiquitous in\nboth research and public perception, but how GANs convert an unstructured\nlatent code to a high quality output is still an open question. In this work,\nwe investigate regression into the latent space as a probe to understand the\ncompositional properties of GANs. We find that combining the regressor and a\npretrained generator provides a strong image prior, allowing us to create\ncomposite images from a collage of random image parts at inference time while\nmaintaining global consistency. To compare compositional properties across\ndifferent generators, we measure the trade-offs between reconstruction of the\nunrealistic input and image quality of the regenerated samples. We find that\nthe regression approach enables more localized editing of individual image\nparts compared to direct editing in the latent space, and we conduct\nexperiments to quantify this independence effect. Our method is agnostic to the\nsemantics of edits, and does not require labels or predefined concepts during\ntraining. Beyond image composition, our method extends to a number of related\napplications, such as image inpainting or example-based image editing, which we\ndemonstrate on several GANs and datasets, and because it uses only a single\nforward pass, it can operate in real-time. Code is available on our project\npage: https://chail.github.io/latent-composition/.\n