Scene relighting with illumination estimation in the latent space on an encoder-decoder scheme

2020/06/03 by Alexandre Pierre Dherse, Dherse, Alexandre Pierre, Martin Everaert +3
Computer Science · #Advanced Vision and Imaging #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Image Enhancement Techniques

paper · pdf · doi:10.48550/arxiv.2006.02333

openalex publication_date 2020/06/03 · openalex created_date 2020/06/12 · openalex updated_date 2026/07/28

Abstract

The image relighting task of transferring illumination conditions between two images offers an interesting and difficult challenge with potential applications in photography, cinematography and computer graphics. In this report we present methods that we tried to achieve that goal. Our models are trained on a rendered dataset of artificial locations with varied scene content, light source location and color temperature. With this dataset, we used a network with illumination estimation component aiming to infer and replace light conditions in the latent space representation of the concerned scenes.

Citations

• The relativistic discriminator: a key element missing from standard GAN
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesis
• VIDIT: Virtual Image Dataset for Illumination Transfer

Related

• Relighting from a Single Image: Datasets and Deep Intrinsic-based Architecture
• LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting
• Controllable Data Augmentation Through Deep Relighting
• Deep Relighting Networks for Image Light Source Manipulation
• Local Relighting of Real Scenes