Multi-Task Imitation Learning for Linear Dynamical Systems

2022/12/01 by Thomas T. Zhang, Katie Kang, Zhang, Thomas T. +11 · 3 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Reinforcement Learning in Robotics #Machine Learning and Algorithms

paper · pdf · doi:10.48550/arxiv.2212.00186

Abstract

We study representation learning for efficient imitation learning over linear systems. In particular, we consider a setting where learning is split into two phases: (a) a pre-training step where a shared k-dimensional representation is learned from H source policies, and (b) a target policy fine-tuning step where the learned representation is used to parameterize the policy class. We find that the imitation gap over trajectories generated by the learned target policy is bounded by O( \frack n_xHN_shared + \frack n_uN_target), where n_x > k is the state dimension, n_u is the input dimension, N_shared denotes the total amount of data collected for each policy during representation learning, and N_target is the amount of target task data. This result formalizes the intuition that aggregating data across related tasks to learn a representation can significantly improve the sample efficiency of learning a target task. The trends suggested by this bound are corroborated in simulation.

Cited by

• Multi-Task GRPO: Reliable LLM Reasoning Across Tasks
• Harnessing Data from Clustered LQR Systems: Personalized and Collaborative Policy Optimization
• Learning clusters of partially observed linear dynamical systems

Related

• A Statistical Guarantee for Representation Transfer in Multitask Imitation Learning
• Provable Representation Learning for Imitation Learning via Bi-level Optimization
• Provable Representation Learning for Imitation with Contrastive Fourier Features
• Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning
• Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysis