Harnessing the Power of Training-Free Techniques in Text-to-2D Generation for Text-to-3D Generation via Score Distillation Sampling

2025/05/26 by Junhong Lee, Seungwook Kim, Lee, Junhong +3
Computer Science · Engineering · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Motion and Animation #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2505.19868

openalex publication_date 2025/05/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recent studies show that simple training-free techniques can dramatically improve the quality of text-to-2D generation outputs, e.g. Classifier-Free Guidance (CFG) or FreeU. However, these training-free techniques have been underexplored in the lens of Score Distillation Sampling (SDS), which is a popular and effective technique to leverage the power of pretrained text-to-2D diffusion models for various tasks. In this paper, we aim to shed light on the effect such training-free techniques have on SDS, via a particular application of text-to-3D generation via 2D lifting. We present our findings, which show that varying the scales of CFG presents a trade-off between object size and surface smoothness, while varying the scales of FreeU presents a trade-off between texture details and geometric errors. Based on these findings, we provide insights into how we can effectively harness training-free techniques for SDS, via a strategic scaling of such techniques in a dynamic manner with respect to the timestep or optimization iteration step. We show that using our proposed scheme strikes a favorable balance between texture details and surface smoothness in text-to-3D generations, while preserving the size of the output and mitigating the occurrence of geometric defects.

Citations

• Understanding the Impact of Negative Prompts: When and How Do They Take Effect?
• CorrespondentDream: Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences
• Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation
• Self-Rectifying Diffusion Sampling with Perturbed-Attention Guidance
• Improving Efficiency of Diffusion Models via Multi-Stage Framework and Tailored Multi-Decoder Architectures
• The Journey, Not the Destination: How Data Guides Diffusion Models
• Diffusing Colors: Image Colorization with Text Guided Diffusion
• Analyzing and Improving the Training Dynamics of Diffusion Models
• ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation
• Intriguing Properties of Data Attribution on Diffusion Models
• Text-to-3D with Classifier Score Distillation
• FreeU: Free Lunch in Diffusion U-Net
• Gradpaint: Gradient-Guided Inpainting with Diffusion Models
• MVDream: Multi-view Diffusion for 3D Generation
• DifFSS: Diffusion Model for Few-Shot Semantic Segmentation
• Emergent Correspondence from Image Diffusion
• Addressing Negative Transfer in Diffusion Models
• ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation
• A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence
• Unsupervised Semantic Correspondence Using Stable Diffusion
• Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence
• TextMesh: Generation of Realistic 3D Meshes From Text Prompts
• Improved Diffusion-based Image Colorization via Piggybacked Models
• Debiasing Scores and Prompts of 2D Diffusion for View-consistent Text-to-3D Generation
• Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation
• Set-the-Scene: Global-Local Training for Generating Controllable NeRF Scenes
• Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions
• Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation
• Multi-Concept Customization of Text-to-Image Diffusion
• Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation
• Latent-NeRF for Shape-Guided Generation of 3D Shapes and Textures
• eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers
• LAION-5B: An open large-scale dataset for training next generation image-text models
• Improving Sample Quality of Diffusion Models Using Self-Attention Guidance
• DreamFusion: Text-to-3D using 2D Diffusion
• An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
• Classifier-Free Diffusion Guidance
• Scaling Autoregressive Models for Content-Rich Text-to-Image Generation
• On Analyzing Generative and Denoising Capabilities of Diffusion-based Deep Generative Models
• Hierarchical Text-Conditional Image Generation with CLIP Latents
• RePaint: Inpainting using Denoising Diffusion Probabilistic Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
• Label-Efficient Semantic Segmentation with Diffusion Models
• Vector Quantized Diffusion Model for Text-to-Image Synthesis
• ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesis
• NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
• SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models
• Image Super-Resolution via Iterative Refinement
• 3D Shape Generation and Completion through Point-Voxel Diffusion
• Learning Transferable Visual Models From Natural Language Supervision
• Score-Based Generative Modeling through Stochastic Differential Equations
• DiffWave: A Versatile Diffusion Model for Audio Synthesis
• WaveGrad: Estimating Gradients for Waveform Generation
• PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
• Text2Shape: Generating Shapes from Natural Language by Learning Joint Embeddings
• Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling

Related

• Text-to-3D with Classifier Score Distillation
• Noise-Free Score Distillation
• Rethinking Score Distilling Sampling for 3D Editing and Generation
• Advancing Text-to-3D Generation with Linearized Lookahead Variational Score Distillation
• VividDreamer: Invariant Score Distillation For Hyper-Realistic Text-to-3D Generation