TopoDiT-3D: Topology-Aware Diffusion Transformer with Bottleneck Structure for 3D Point Cloud Generation

2025/05/14 by Guan, Zechao, Yan Feng, Shuai Du +6
Computer Science · Earth and Planetary Sciences · Engineering · #3D Shape Modeling and Analysis #3D Surveying and Cultural Heritage #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · pdf · doi:10.48550/arxiv.2505.09140

openalex publication_date 2025/05/14 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent advancements in Diffusion Transformer (DiT) models have significantly improved 3D point cloud generation. However, existing methods primarily focus on local feature extraction while overlooking global topological information, such as voids, which are crucial for maintaining shape consistency and capturing complex geometries. To address this limitation, we propose TopoDiT-3D, a Topology-Aware Diffusion Transformer with a bottleneck structure for 3D point cloud generation. Specifically, we design the bottleneck structure utilizing Perceiver Resampler, which not only offers a mode to integrate topological information extracted through persistent homology into feature learning, but also adaptively filters out redundant local features to improve training efficiency. Experimental results demonstrate that TopoDiT-3D outperforms state-of-the-art models in visual quality, diversity, and training efficiency. Furthermore, TopoDiT-3D demonstrates the importance of rich topological information for 3D point cloud generation and its synergy with conventional local feature learning. Videos and code are available at https://github.com/Zechao-Guan/TopoDiT-3D.

Citations

• RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulation
• Vision-Language Foundation Models as Effective Robot Imitators
• Wonder3D: Single Image to 3D using Cross-Domain Diffusion
• OpenFlamingo: An Open-Source Framework for Training Large Autoregressive Vision-Language Models
• DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation
• Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models
• MultiModal-GPT: A Vision and Language Model for Dialogue with Humans
• MDTv2: Masked Diffusion Transformer is a Strong Image Synthesizer
• Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior
• MeshDiffusion: Score-based Generative 3D Mesh Modeling
• Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models
• Scalable Diffusion Models with Transformers
• LION: Latent Point Diffusion Models for 3D Shape Generation
• GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images
• AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generation
• GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
• CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation
• Diffusion Models Beat GANs on Image Synthesis
• GANcraft: Unsupervised 3D Neural Rendering of Minecraft Worlds
• 3D Shape Generation and Completion through Point-Voxel Diffusion
• SetVAE: Learning Hierarchical Composition for Generative Modeling of Set-Structured Data
• Diffusion Probabilistic Models for 3D Point Cloud Generation
• Discrete Point Flow Networks for Efficient Point Cloud Generation
• SoftFlow: Probabilistic Framework for Normalizing Flow on Manifolds
• C-Flow: Conditional Generative Flow Models for Images and 3D Point Clouds
• Persistence Homology of Networks: Methods and Applications
• PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
• A Topology Layer for Machine Learning
• 3D Point Cloud Generative Adversarial Network Based on Tree Structured Graph Convolutions
• Multiresolution Tree Networks for 3D Point Cloud Processing
• Deformable Shape Completion with Graph Convolutional Autoencoders
• Variational Autoencoders for Deforming 3D Mesh Models
• Attention Is All You Need
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
• Auto-Encoding Variational Bayes

Related

• DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generation
• Fast Training of Diffusion Transformer with Extreme Masking for 3D Point Clouds Generation
• Topology-Aware Latent Diffusion for 3D Shape Generation
• TopoDiffusionNet: A Topology-aware Diffusion Model
• DiffTF++: 3D-aware Diffusion Transformer for Large-Vocabulary 3D Generation