MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Model

2024/04/30 by Wenxun Dai, Dai, Wenxun, Ling-Hao Chen +9 · 33 citations
Computer Science · Engineering · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Motion and Animation #Human Pose and Action Recognition #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2404.19759

openalex publication_date 2024/04/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This work introduces MotionLCM, extending controllable motion generation to a real-time level. Existing methods for spatial-temporal control in text-conditioned motion generation suffer from significant runtime inefficiency. To address this issue, we first propose the motion latent consistency model (MotionLCM) for motion generation, building on the motion latent diffusion model. By adopting one-step (or few-step) inference, we further improve the runtime efficiency of the motion latent diffusion model for motion generation. To ensure effective controllability, we incorporate a motion ControlNet within the latent space of MotionLCM and enable explicit control signals (i.e., initial motions) in the vanilla motion space to further provide supervision for the training process. By employing these techniques, our approach can generate human motions with text and control signals in real-time. Experimental results demonstrate the remarkable generation and controlling capabilities of MotionLCM while maintaining real-time runtime efficiency.

Cited by

• HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generation
• Towards Arbitrary Motion Completing via Hierarchical Continuous Representation
• MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation
• A Survey of Body and Face Motion: Datasets, Performance Evaluation Metrics and Generative Techniques
• CoordSpeaker: Exploiting Gesture Captioning for Coordinated Caption-Empowered Co-Speech Gesture Generation
• ReAlign: Text-to-Motion Generation via Step-Aware Reward-Guided Alignment
• Free3D: 3D Human Motion Emerges from Single-View 2D Supervision
• Human Motion Synthesis in 3D Scenes via Unified Scene Semantic Occupancy
• Pressure2Motion: Hierarchical Human Motion Reconstruction from Ground Pressure with Text Guidance
• The Quest for Generalizable Motion Generation: Data, Model, and Evaluation
• Endowing GPT-4 with a Humanoid Body: Building the Bridge Between Off-the-Shelf VLMs and the Physical World
• LILAC: Long-sequence Incremental Low-latency Arbitrary Motion Stylization via Streaming VAE-Diffusion with Causal Decoding
• HumanCM: One Step Human Motion Prediction
• Generative human motion mimicking through feature extraction in denoising diffusion settings
• SceneAdapt: Scene-aware Adaptation of Human Motion Diffusion
• Learning Human Motion with Temporally Conditional Mamba
• FastHMR: Accelerating Human Mesh Recovery via Token and Layer Merging with Diffusion Decoding
• Pulp Motion: Framing-aware multimodal camera and human motion generation
• LUMA: Low-Dimension Unified Motion Alignment with Dual-Path Anchoring for Text-to-Motion Diffusion Model
• Unified Multi-Modal Interactive & Reactive 3D Motion Generation via Rectified Flow
• Learning to Understand Body Language from Flight through Robust 3D Avatar Placing
• MUGEN: A Unified Framework for Efficient Motion Understanding and Generation
• ScoreHOI: Physically Plausible Reconstruction of Human-Object Interaction via Score-Guided Diffusion
• InterPose: Learning to Generate Human-Object Interactions from Large-Scale Web Videos
• Embracing Aleatoric Uncertainty: Generating Diverse 3D Human Motion
• MotionFlux: Efficient Text-Guided Motion Generation through Rectified Flow Matching and Preference Alignment
• Scalable RF Simulation in Generative 4D Worlds
• VimoRAG: Video-based Retrieval-augmented 3D Motion Generation for Motion Language Models
• Spatial-Temporal Multi-Scale Quantization for Flexible Motion Generation
• PP-Motion: Physical-Perceptual Fidelity Evaluation for Human Motion Generation
• X-MoGen: Unified Motion Generation across Humans and Animals
• Thinking With Videos: Multimodal Tool-Augmented Reinforcement Learning for Long Video Reasoning
• PINO: Person-Interaction Noise Optimization for Long-Duration and Customizable Motion Generation of Arbitrary-Sized Groups

Related

• MotionPCM: Real-Time Motion Synthesis with Phased Consistency Model
• DartControl: A Diffusion-Based Autoregressive Motion Model for Real-Time Text-Driven Motion Control
• Executing your Commands via Motion Diffusion in Latent Space
• MotionGPT: Finetuned LLMs Are General-Purpose Motion Generators
• MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model