Sel3DCraft: Interactive Visual Prompts for User-Friendly Text-to-3D Generation

2025/08/01 by Xiang, Nan, Liang, Tianyi, Huang, Haiwen +6
#FOS: Computer and information sciences #Graphics (cs.GR) #Human-Computer Interaction (cs.HC)

paper · doi:10.48550/arxiv.2508.00428

Abstract

Text-to-3D (T23D) generation has transformed digital content creation, yet remains bottlenecked by blind trial-and-error prompting processes that yield unpredictable results. While visual prompt engineering has advanced in text-to-image domains, its application to 3D generation presents unique challenges requiring multi-view consistency evaluation and spatial understanding. We present Sel3DCraft, a visual prompt engineering system for T23D that transforms unstructured exploration into a guided visual process. Our approach introduces three key innovations: a dual-branch structure combining retrieval and generation for diverse candidate exploration; a multi-view hybrid scoring approach that leverages MLLMs with innovative high-level metrics to assess 3D models with human-expert consistency; and a prompt-driven visual analytics suite that enables intuitive defect identification and refinement. Extensive testing and user studies demonstrate that Sel3DCraft surpasses other T23D systems in supporting creativity for designers.

Citations

• REPARO: Compositional 3D Assets Generation with Differentiable 3D Layout Alignment
• ComboVerse: Compositional 3D Assets Creation Using Spatially-Aware Diffusion Guidance
• GVGEN: Text-to-3D Generation with Volumetric Representation
• MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark
• LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation
• GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
• VolumeDiffusion: Flexible Text-to-3D Generation with Efficient Volumetric Encoder
• Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model
• LRM: Large Reconstruction Model for Single Image to 3D
• Visual Analytics for Efficient Image Exploration and User-Guided Image Captioning
• Wonder3D: Single Image to 3D using Cross-Domain Diffusion
• GenQuery: Supporting Expressive Visual Search with Generative Models
• DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation
• MVDream: Multi-view Diffusion for 3D Generation
• Objaverse-XL: A Universe of 10M+ 3D Objects
• SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
• Michelangelo: Conditional 3D Shape Generation based on Shape-Image-Text Aligned Latent Representation
• Zero-Shot 3D Shape Correspondence
• OpenShape: Scaling Up 3D Shape Representation Towards Open-World Understanding
• Promptify: Text-to-Image Generation through Interactive Prompt Exploration with Large Language Models
• Micrograph segmentations for DDEVD
• Segment Anything
• Zero-1-to-3: Zero-shot One Image to 3D Object
• Point-E: A System for Generating 3D Point Clouds from Complex Prompts
• Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation
• DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models
• 3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows
• LION: Latent Point Diffusion Models for 3D Shape Generation
• DreamFusion: Text-to-3D using 2D Diffusion
• Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models
• MultiViz: Towards Visualizing and Understanding Multimodal Models
• DendroMap: Visual Exploration of Large-Scale Image Datasets for Machine Learning with Treemaps
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• PromptChainer: Chaining Large Language Model Prompts through Visual\n Programming
• Training language models to follow instructions with human feedback
• BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
• Design Guidelines for Prompt Engineering Text-to-Image Generative Models
• Learning Transferable Visual Models From Natural Language Supervision
• Making Pre-trained Language Models Better Few-shot Learners
• Language Models are Few-Shot Learners
• Quantifying the Creativity Support of Digital Tools through the Creativity Support Index
• GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling

Related

• TV-3DG: Mastering Text-to-3D Customized Generation with Visual Prompt
• 3DALL-E: Integrating Text-to-Image AI in 3D Design Workflows
• Gen3DEval: Using vLLMs for Automatic Evaluation of Generated 3D Objects
• VP3D: Unleashing 2D Visual Prompt for Text-to-3D Generation
• T³Bench: Benchmarking Current Progress in Text-to-3D Generation