2022/07/01 by Tiancheng Zhi, Bowei Chen, Ivaylo Boyadzhiev +3
Computer Science · Environmental Science · #Advanced Vision and Imaging #Image Enhancement Techniques #Remote Sensing and LiDAR Applications
paper · doi:10.1145/3528223.3530148
openalex publication_date 2022/07/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/06/11
We describe a novel approach to decompose a single panorama of an empty indoor environment into four appearance components: specular, direct sunlight, diffuse and diffuse ambient without direct sunlight. Our system is weakly supervised by automatically generated semantic maps (with floor, wall, ceiling, lamp, window and door labels) that have shown success on perspective views and are trained for panoramas using transfer learning without any further annotations. A GAN-based approach supervised by coarse information obtained from the semantic map extracts specular reflection and direct sunlight regions on the floor and walls. These lighting effects are removed via a similar GAN-based approach and a semantic-aware inpainting step. The appearance decomposition enables multiple applications including sun direction estimation, virtual furniture insertion, floor material replacement, and sun direction change, providing an effective tool for virtual home staging. We demonstrate the effectiveness of our approach on a large and recently released dataset of panoramas of empty homes.