Knowledge Transfer in Model-Based Reinforcement Learning Agents for Efficient Multi-Task Learning

2025/01/09 by Dmytro Kuzmenko, Kuzmenko, Dmytro, Nadiya Shvai +1 · 1 citation
Computer Science · #FOS: Computer and information sciences #I.2.11 #I.2.9 #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2501.05329

openalex publication_date 2025/01/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We propose an efficient knowledge transfer approach for model-based reinforcement learning, addressing the challenge of deploying large world models in resource-constrained environments. Our method distills a high-capacity multi-task agent (317M parameters) into a compact 1M parameter model, achieving state-of-the-art performance on the MT30 benchmark with a normalized score of 28.45, a substantial improvement over the original 1M parameter model's score of 18.93. This demonstrates the ability of our distillation technique to consolidate complex multi-task knowledge effectively. Additionally, we apply FP16 post-training quantization, reducing the model size by 50% while maintaining performance. Our work bridges the gap between the power of large models and practical deployment constraints, offering a scalable solution for efficient and accessible multi-task reinforcement learning in robotics and other resource-limited domains.

Cited by

• TD-MPC-Opt: Distilling Model-Based Multi-Task Reinforcement Learning Agents

Related

• TD-MPC-Opt: Distilling Model-Based Multi-Task Reinforcement Learning Agents
• On the Feasibility of Cross-Task Transfer with Model-Based Reinforcement Learning
• Multi-Source Transfer Learning for Deep Model-Based Reinforcement Learning
• Investigating the role of model-based learning in exploration and transfer
• Model Evolution Framework with Genetic Algorithm for Multi-Task Reinforcement Learning