Atlas Gaussians Diffusion for 3D Generation

2024/08/23 by Haitao Yang, Yang, Haitao, Yuan Dong +9 · 7 citations
Computer Science · Engineering · Medicine · #3D Shape Modeling and Analysis #Advanced Vision and Imaging #Artificial intelligence #Atlas (anatomy) #Cartography #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Diffusion #Diffusion MRI #FOS: Computer and information sciences #Geography #Geology #Medicine #Mixture model #Paleontology #Physics #Statistical physics #Thermodynamics

paper · pdf · doi:10.48550/arxiv.2408.13055

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2024/08/23 · openalex created_date 2024/09/21 · openalex updated_date 2026/08/06

Abstract

Using the latent diffusion model has proven effective in developing novel 3D generation techniques. To harness the latent diffusion model, a key challenge is designing a high-fidelity and efficient representation that links the latent space and the 3D space. In this paper, we introduce Atlas Gaussians, a novel representation for feed-forward native 3D generation. Atlas Gaussians represent a shape as the union of local patches, and each patch can decode 3D Gaussians. We parameterize a patch as a sequence of feature vectors and design a learnable function to decode 3D Gaussians from the feature vectors. In this process, we incorporate UV-based sampling, enabling the generation of a sufficiently large, and theoretically infinite, number of 3D Gaussian points. The large amount of 3D Gaussians enables the generation of high-quality details. Moreover, due to local awareness of the representation, the transformer-based decoding procedure operates on a patch level, ensuring efficiency. We train a variational autoencoder to learn the Atlas Gaussians representation, and then apply a latent diffusion model on its latent space for learning 3D Generation. Experiments show that our approach outperforms the prior arts of feed-forward native 3D generation. Project page: https://yanghtr.github.io/projects/atlas_gaussians.

Cited by

• Native and Compact Structured Latents for 3D Generation
• Repurposing 2D Diffusion Models for 3D Shape Completion
• GaussianBlender: Instant Stylization of 3D Gaussians with Disentangled Latent Spaces
• UniLat3D: Geometry-Appearance Unified Latents for Single-Stage 3D Generation
• Artist-Created Mesh Generation from Raw Observation
• A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation
• Gaussian Variation Field Diffusion for High-fidelity Video-to-4D Synthesis

Related

• Repurposing 2D Diffusion Models with Gaussian Atlas for 3D Generation
• GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models
• Sampling 3D Gaussian Scenes in Seconds with Latent Diffusion Models
• Representing 3D Shapes With 64 Latent Vectors for 3D Diffusion Models
• LT3SD: Latent Trees for 3D Scene Diffusion