Zero-Shot Policy Transfer in Reinforcement Learning using Buckingham's Pi Theorem

2025/10/09 by Francisco Pascoa, Pascoa, Francisco, Ian Lalonde +3 · 1 citation
Computer Science · Engineering · #Adaptive Dynamic Programming Control #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Robot Manipulation and Learning #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2510.08768

openalex publication_date 2025/10/09 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28

Abstract

Reinforcement learning (RL) policies often fail to generalize to new robots, tasks, or environments with different physical parameters, a challenge that limits their real-world applicability. This paper presents a simple, zero-shot transfer method based on Buckingham's Pi Theorem to address this limitation. The method adapts a pre-trained policy to new system contexts by scaling its inputs (observations) and outputs (actions) through a dimensionless space, requiring no retraining. The approach is evaluated against a naive transfer baseline across three environments of increasing complexity: a simulated pendulum, a physical pendulum for sim-to-real validation, and the high-dimensional HalfCheetah. Results demonstrate that the scaled transfer exhibits no loss of performance on dynamically similar contexts. Furthermore, on non-similar contexts, the scaled policy consistently outperforms the naive transfer, significantly expanding the volume of contexts where the original policy remains effective. These findings demonstrate that dimensional analysis provides a powerful and practical tool to enhance the robustness and generalization of RL policies.

Citations

• Improving Controller Generalization with Dimensionless Markov Decision Processes
• Leveraging Symmetry to Accelerate Learning of Trajectory Tracking Controllers for Free-Flying Robotic Systems
• Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges
• Transfer Learning in Deep Reinforcement Learning: A Survey
• Controlling Overestimation Bias with Truncated Mixture of Continuous Distributional Quantile Critics
• Lagrangian Neural Networks
• CrossQ: Batch Normalization in Deep Reinforcement Learning for Greater Sample Efficiency and Simplicity
• Addressing Function Approximation Error in Actor-Critic Methods
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Cited by

• Direct transfer of optimized controllers to similar systems using dimensionless MPC

Related

• Pessimism Principle Can Be Effective: Towards a Framework for Zero-Shot Transfer Reinforcement Learning
• Zero-Shot Policy Transfer with Disentangled Task Representation of Meta-Reinforcement Learning
• VPE: Variational Policy Embedding for Transfer Reinforcement Learning
• Adaptive Policy Transfer in Reinforcement Learning
• Efficient Reuse of Previous Experiences to Improve Policies in Real Environment