ReinDiffuse: Crafting Physically Plausible Motions with Reinforced Diffusion Model

2024/10/09 by Gaoge Han, Han, Gaoge, Mingjiang Liang +9 · 6 citations
Engineering · #Additive Manufacturing and 3D Printing Technologies #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Innovations in Concrete and Construction Materials #Manufacturing Process and Optimization

paper · pdf · doi:10.48550/arxiv.2410.07296

openalex publication_date 2024/10/09 · openalex created_date 2024/10/13 · openalex updated_date 2026/07/28

Abstract

Generating human motion from textual descriptions is a challenging task. Existing methods either struggle with physical credibility or are limited by the complexities of physics simulations. In this paper, we present ReinDiffuse that combines reinforcement learning with motion diffusion model to generate physically credible human motions that align with textual descriptions. Our method adapts Motion Diffusion Model to output a parameterized distribution of actions, making them compatible with reinforcement learning paradigms. We employ reinforcement learning with the objective of maximizing physically plausible rewards to optimize motion generation for physical fidelity. Our approach outperforms existing state-of-the-art models on two major datasets, HumanML3D and KIT-ML, achieving significant improvements in physical plausibility and motion quality. Project: https://reindiffuse.github.io/

Cited by

• Uni-Inter: Unifying 3D Human Motion Synthesis Across Diverse Interaction Contexts
• ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignment
• No MoCap Needed: Post-Training Motion Diffusion Models with Reinforcement Learning using Only Textual Prompts
• SimDiff: Simulator-constrained Diffusion Model for Physically Plausible Motion Generation
• PP-Motion: Physical-Perceptual Fidelity Evaluation for Human Motion Generation
• Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos

Related

• No MoCap Needed: Post-Training Motion Diffusion Models with Reinforcement Learning using Only Textual Prompts
• PhysDiff: Physics-Guided Human Motion Diffusion Model
• Generation of Complex 3D Human Motion by Temporal and Spatial Composition of Diffusion Models
• Realistic Human Motion Generation with Cross-Diffusion Models
• MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model