2023/12/13 by Thomas Tanay, Tanay, Thomas, Matteo Maggioni +1 · 2 citations
Computer Science · #3D computer graphics #3D rendering #Advanced Vision and Imaging #Artificial intelligence #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #Computer graphics #Computer graphics (images) #Computer science #Computer vision #Convolutional neural network #FOS: Computer and information sciences #Global illumination #Image Enhancement Techniques #Image-based modeling and rendering #Pixel #Real-time rendering #Rendering (computer graphics) #Software rendering #Tiled rendering #Volume rendering
paper · pdf · doi:10.48550/arxiv.2312.08338
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2023/12/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
A recent trend among generalizable novel view synthesis methods is to learn a rendering operator acting over single camera rays. This approach is promising because it removes the need for explicit volumetric rendering, but it effectively treats target images as collections of independent pixels. Here, we propose to learn a global rendering operator acting over all camera rays jointly. We show that the right representation to enable such rendering is a 5-dimensional plane sweep volume consisting of the projection of the input images on a set of planes facing the target camera. Based on this understanding, we introduce our Convolutional Global Latent Renderer (ConvGLR), an efficient convolutional architecture that performs the rendering operation globally in a low-resolution latent space. Experiments on various datasets under sparse and generalizable setups show that our approach consistently outperforms existing methods by significant margins.