2019/12/16 by Michael Niemeyer, Lars Mescheder, Niemeyer, Michael +5 · 18 citations
Computer Science · Engineering · #3D Shape Modeling and Analysis #Advanced Vision and Imaging #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.1912.07372
openalex publication_date 2019/12/16 · openalex created_date 2022/07/26 · openalex updated_date 2026/07/28
Learning-based 3D reconstruction methods have shown impressive results.\nHowever, most methods require 3D supervision which is often hard to obtain for\nreal-world datasets. Recently, several works have proposed differentiable\nrendering techniques to train reconstruction models from RGB images.\nUnfortunately, these approaches are currently restricted to voxel- and\nmesh-based representations, suffering from discretization or low resolution. In\nthis work, we propose a differentiable rendering formulation for implicit shape\nand texture representations. Implicit representations have recently gained\npopularity as they represent shape and texture continuously. Our key insight is\nthat depth gradients can be derived analytically using the concept of implicit\ndifferentiation. This allows us to learn implicit shape and texture\nrepresentations directly from RGB images. We experimentally show that our\nsingle-view reconstructions rival those learned with full 3D supervision.\nMoreover, we find that our method can be used for multi-view 3D reconstruction,\ndirectly resulting in watertight meshes.\n