U-DiT Policy: U-shaped Diffusion Transformers for Robotic Manipulation

2025/09/29 by Wu, Linzhi, Mei, Aoran, Wang, Xiyue +2
#FOS: Computer and information sciences #Robotics (cs.RO)

paper · doi:10.48550/arxiv.2509.24579

Abstract

Diffusion-based methods have been acknowledged as a powerful paradigm for end-to-end visuomotor control in robotics. Most existing approaches adopt a Diffusion Policy in U-Net architecture (DP-U), which, while effective, suffers from limited global context modeling and over-smoothing artifacts. To address these issues, we propose U-DiT Policy, a novel U-shaped Diffusion Transformer framework. U-DiT preserves the multi-scale feature fusion advantages of U-Net while integrating the global context modeling capability of Transformers, thereby enhancing representational power and policy expressiveness. We evaluate U-DiT extensively across both simulation and real-world robotic manipulation tasks. In simulation, U-DiT achieves an average performance gain of 10% over baseline methods and surpasses Transformer-based diffusion policies (DP-T) that use AdaLN blocks by 6% under comparable parameter budgets. On real-world robotic tasks, U-DiT demonstrates superior generalization and robustness, achieving an average improvement of 22.5% over DP-U. In addition, robustness and generalization experiments under distractor and lighting variations further highlight the advantages of U-DiT. These results highlight the effectiveness and practical potential of U-DiT Policy as a new foundation for diffusion-based robotic manipulation.

Citations

• π₀: A Vision-Language-Action Flow Model for General Robot Control
• WorldSimBench: Towards Video Generation Models as World Simulators
• The Ingredients for Robotic Diffusion Transformers
• Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic Manipulation
• DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flow
• Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
• Diffusion policy: Visuomotor policy learning via action diffusion
• Scaling Robot Learning with Semantically Imagined Experience
• Simple diffusion: End-to-end diffusion for high resolution images
• Scalable Diffusion Models with Transformers
• Speech Enhancement and Dereverberation with Diffusion-based Generative Models
• Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
• Implicit Behavioral Cloning
• What Matters in Learning from Offline Human Demonstrations for Robot Manipulation
• Denoising Diffusion Implicit Models
• Denoising Diffusion Probabilistic Models
• Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains
• Improved Techniques for Training Score-Based Generative Models
• End-to-End Object Detection with Transformers
• Scaling Laws for Neural Language Models
• RLBench: The Robot Learning Benchmark & Learning Environment
• FiLM: Visual Reasoning with a General Conditioning Layer
• Attention Is All You Need
• U-Net: Convolutional Networks for Biomedical Image Segmentation
• End-to-End Training of Deep Visuomotor Policies
• A survey of robot learning from demonstration

Related

• Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic Manipulation
• Diffusion Transformer Policy
• MTDP: A Modulated Transformer based Diffusion Policy Model
• The Ingredients for Robotic Diffusion Transformers
• Diffusion Policy: Visuomotor Policy Learning via Action Diffusion