2019/09/03 by Yida Wang, Wang, Yida, David Joseph Tan +5 · 1 citation
Computer Science · Engineering · #3D Shape Modeling and Analysis #Advanced Vision and Imaging #Artificial Intelligence (cs.AI) #Computational Geometry (cs.CG) #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Electrical engineering #Image and Video Processing (eess.IV) #Machine Learning (cs.LG) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.1909.01106
openalex publication_date 2019/09/03 · openalex created_date 2019/09/12 · openalex updated_date 2026/07/28
We propose a novel model for 3D semantic completion from a single depth image, based on a single encoder and three separate generators used to reconstruct different geometric and semantic representations of the original and completed scene, all sharing the same latent space. To transfer information between the geometric and semantic branches of the network, we introduce paths between them concatenating features at corresponding network layers. Motivated by the limited amount of training samples from real scenes, an interesting attribute of our architecture is the capacity to supplement the existing dataset by generating a new training dataset with high quality, realistic scenes that even includes occlusion and real noise. We build the new dataset by sampling the features directly from latent space which generates a pair of partial volumetric surface and completed volumetric semantic surface. Moreover, we utilize multiple discriminators to increase the accuracy and realism of the reconstructions. We demonstrate the benefits of our approach on standard benchmarks for the two most common completion tasks: semantic 3D scene completion and 3D object completion.