GeoDiffusion: A Training-Free Framework for Accurate 3D Geometric Conditioning in Image Generation

2025/10/25 by Mueller, Phillip, Uenlue, Talip, Schmidt, Sebastian +4
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2510.22337

Abstract

Precise geometric control in image generation is essential for engineering & product design and creative industries to control 3D object features accurately in image space. Traditional 3D editing approaches are time-consuming and demand specialized skills, while current image-based generative methods lack accuracy in geometric conditioning. To address these challenges, we propose GeoDiffusion, a training-free framework for accurate and efficient geometric conditioning of 3D features in image generation. GeoDiffusion employs a class-specific 3D object as a geometric prior to define keypoints and parametric correlations in 3D space. We ensure viewpoint consistency through a rendered image of a reference 3D object, followed by style transfer to meet user-defined appearance specifications. At the core of our framework is GeoDrag, improving accuracy and speed of drag-based image editing on geometry guidance tasks and general instructions on DragBench. Our results demonstrate that GeoDiffusion enables precise geometric modifications across various iterative design workflows.

Citations

• MvDrag3D: Drag-based Creative 3D Editing via Multi-view Generation-Reconstruction Priors
• 3DRealCar: An In-the-wild RGB-D Car Dataset with 360-degree Views
• GoodDrag: Towards Good Practices for Drag Editing with Diffusion Models
• Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation
• SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
• DrivAerNet: A Parametric Car Dataset for Data-Driven Aerodynamic Design and Prediction
• DiffEditor: Boosting Accuracy and Flexibility on Diffusion-based Image Editing
• Image Sculpting: Precise Object Editing with 3D Geometry Control
• LRM: Large Reconstruction Model for Single Image to 3D
• The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing
• Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model
• Predict, Refine, Synthesize: Self-Guiding Diffusion Models for Probabilistic Time Series Forecasting
• FreeDrag: Feature Dragging for Reliable Point-based Image Editing
• DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models
• SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
• DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing
• Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis
• Generating Images with 3D Annotations Using Diffusion Models
• A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence
• Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence
• Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold
• Zero-1-to-3: Zero-shot One Image to 3D Object
• T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models
• Adding Conditional Control to Text-to-Image Diffusion Models
• Adding Conditional Control to Text-to-Image Diffusion Models
• OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation
• Objaverse: A Universe of Annotated 3D Objects
• Sketch-Guided Text-to-Image Diffusion Models
• Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation
• InstructPix2Pix: Learning to Follow Image Editing Instructions
• Imagic: Text-Based Real Image Editing with Diffusion Models
• Prompt-to-Prompt Image Editing with Cross Attention Control
• Pretraining is All You Need for Image-to-Image Translation
• RePaint: Inpainting using Denoising Diffusion Probabilistic Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• Deep ViT Features as Dense Visual Descriptors
• Palette: Image-to-Image Diffusion Models
• SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
• LoRA: Low-Rank Adaptation of Large Language Models
• Diffusion Models Beat GANs on Image Synthesis
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Denoising Diffusion Implicit Models
• Denoising Diffusion Probabilistic Models
• The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
• ShapeNet: An Information-Rich 3D Model Repository
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• Deep Unsupervised Learning using Nonequilibrium Thermodynamics
• Image quality assessment: from error visibility to structural similarity

Related

• GeoDiffuser: Geometry-Based Image Editing with Diffusion Models
• Dragging with Geometry: From Pixels to Geometry-Guided Image Editing
• Training-free Geometric Image Editing on Diffusion Models
• FreeInsert: Personalized Object Insertion with Geometric and Style Control
• GeoRemover: Removing Objects and Their Causal Visual Artifacts