RMD: A Simple Baseline for More General Human Motion Generation via Training-free Retrieval-Augmented Motion Diffuse

2024/12/05 by Liao, Zhouyingcheng, Zhang, Mingyuan, Wang, Wenjia +2 · 2 citations
#Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Graphics (cs.GR)

paper · doi:10.48550/arxiv.2412.04343

Abstract

While motion generation has made substantial progress, its practical application remains constrained by dataset diversity and scale, limiting its ability to handle out-of-distribution scenarios. To address this, we propose a simple and effective baseline, RMD, which enhances the generalization of motion generation through retrieval-augmented techniques. Unlike previous retrieval-based methods, RMD requires no additional training and offers three key advantages: (1) the external retrieval database can be flexibly replaced; (2) body parts from the motion database can be reused, with an LLM facilitating splitting and recombination; and (3) a pre-trained motion diffusion model serves as a prior to improve the quality of motions obtained through retrieval and direct combination. Without any training, RMD achieves state-of-the-art performance, with notable advantages on out-of-distribution data.

Cited by

• IRG-MotionLLM: Interleaving Motion Generation, Assessment and Refinement for Text-to-Motion Generation
• Unified Multi-Modal Interactive & Reactive 3D Motion Generation via Rectified Flow
• ReMoMask: Retrieval-Augmented Masked Motion Generation

Related

• ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
• MoRAG -- Multi-Fusion Retrieval Augmented Generation for Human Motion
• ReMoMask: Retrieval-Augmented Masked Motion Generation
• No MoCap Needed: Post-Training Motion Diffusion Models with Reinforcement Learning using Only Textual Prompts
• FG-MDM: Towards Zero-Shot Human Motion Generation via ChatGPT-Refined Descriptions