2022/11/30 by J. Ryan Shue, Shue, J. Ryan, Eric Ryan Chan +9 · 43 citations
Computer Science · Engineering · Mathematics · #3D Shape Modeling and Analysis #Algorithm #Artificial intelligence #Benchmark (surveying) #Computer Graphics and Visualization Techniques #Computer graphics (images) #Computer science #Diffusion #Factoring #Feature (linguistics) #Field (mathematics) #Generative Adversarial Networks and Image Synthesis #Geology #Mathematics #Object (grammar) #Pattern recognition (psychology) #Polygon mesh #Set (abstract data type)
paper · pdf · doi:10.48550/arxiv.2211.16677
published in arXiv (Cornell University) (Cornell University)
openalex publication_date 2022/11/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Diffusion models have emerged as the state-of-the-art for image generation, among other tasks. Here, we present an efficient diffusion-based model for 3D-aware generation of neural fields. Our approach pre-processes training data, such as ShapeNet meshes, by converting them to continuous occupancy fields and factoring them into a set of axis-aligned triplane feature representations. Thus, our 3D training scenes are all represented by 2D feature planes, and we can directly train existing 2D diffusion models on these representations to generate 3D neural fields with high quality and diversity, outperforming alternative approaches to 3D-aware generation. Our approach requires essential modifications to existing triplane factorization pipelines to make the resulting features easy to learn for the diffusion model. We demonstrate state-of-the-art results on 3D generation on several object classes from ShapeNet.