StableMoFusion: Towards Robust and Efficient Diffusion-based Motion Generation Framework

2024/05/09 by Yiheng Huang, Hui Yang, Huang, Yiheng +13 · 23 citations
Computer Science · Engineering · #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Motion and Animation #Human Pose and Action Recognition #Multimedia (cs.MM)

paper · pdf · doi:10.48550/arxiv.2405.05691

openalex publication_date 2024/05/09 · openalex created_date 2024/05/11 · openalex updated_date 2026/07/28

Abstract

Thanks to the powerful generative capacity of diffusion models, recent years have witnessed rapid progress in human motion generation. Existing diffusion-based methods employ disparate network architectures and training strategies. The effect of the design of each component is still unclear. In addition, the iterative denoising process consumes considerable computational overhead, which is prohibitive for real-time scenarios such as virtual characters and humanoid robots. For this reason, we first conduct a comprehensive investigation into network architectures, training strategies, and inference processs. Based on the profound analysis, we tailor each component for efficient high-quality human motion generation. Despite the promising performance, the tailored model still suffers from foot skating which is an ubiquitous issue in diffusion-based solutions. To eliminate footskate, we identify foot-ground contact and correct foot motions along the denoising process. By organically combining these well-designed components together, we present StableMoFusion, a robust and efficient framework for human motion generation. Extensive experimental results show that our StableMoFusion performs favorably against current state-of-the-art methods. Project page: https://h-y1heng.github.io/StableMoFusion-page/

Cited by

• Interact2Ar: Full-Body Human-Human Interaction Generation via Autoregressive Diffusion Models
• MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation
• RoleMotion: A Large-Scale Dataset towards Robust Scene-Specific Role-Playing Motion Synthesis with Fine-grained Descriptions
• No MoCap Needed: Post-Training Motion Diffusion Models with Reinforcement Learning using Only Textual Prompts
• LUMA: Low-Dimension Unified Motion Alignment with Dual-Path Anchoring for Text-to-Motion Diffusion Model
• SIG-Chat: Spatial Intent-Guided Conversational Gesture Generation Involving How, When and Where
• MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
• Geometric Neural Distance Fields for Learning Human Motion Priors
• VimoRAG: Video-based Retrieval-augmented 3D Motion Generation for Motion Language Models
• PP-Motion: Physical-Perceptual Fidelity Evaluation for Human Motion Generation
• PA-HOI: A Physics-Aware Human and Object Interaction Dataset
• X-MoGen: Unified Motion Generation across Humans and Animals
• Inversion-DPO: Precise and Efficient Post-Training for Diffusion Models
• SnapMoGen: Human Motion Generation from Expressive Texts
• Go to Zero: Towards Zero-shot Motion Generation with Million-scale Data
• ANT: Adaptive Neural Temporal-Aware Text-to-Motion Model
• Absolute Coordinates Make Motion Generation Easy
• GenPrior: Unleashing Text-to-Motion Generative Priors for Zero-Shot Skeleton-based Action Recognition
• Towards Robust and Controllable Text-to-Motion via Masked Autoregressive Diffusion
• Text-driven Motion Generation: Overview, Challenges and Directions
• Causal Motion Diffusion Models for Autoregressive Motion Generation
• Efficient Listener: Dyadic Facial Motion Synthesis via Action Diffusion
• PMG: Progressive Motion Generation via Sparse Anchor Postures Curriculum Learning

Related

• Human Motion Diffusion Model
• MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation
• MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis
• EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation
• Strong and Controllable 3D Motion Generation