Physics-Informed Model and Hybrid Planning for Efficient Dyna-Style Reinforcement Learning

2024/07/02 by Zakariae El Asri, Olivier Sigaud, Asri, Zakariae El +3 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2407.02217

openalex publication_date 2024/07/02 · openalex created_date 2024/07/06 · openalex updated_date 2026/07/28

Abstract

Applying reinforcement learning (RL) to real-world applications requires addressing a trade-off between asymptotic performance, sample efficiency, and inference time. In this work, we demonstrate how to address this triple challenge by leveraging partial physical knowledge about the system dynamics. Our approach involves learning a physics-informed model to boost sample efficiency and generating imaginary trajectories from this model to learn a model-free policy and Q-function. Furthermore, we propose a hybrid planning strategy, combining the learned policy and Q-function with the learned model to enhance time efficiency in planning. Through practical demonstrations, we illustrate that our method improves the compromise between sample efficiency, time efficiency, and performance over state-of-the-art methods.

Cited by

• RT-HCP: Dealing with Inference Delays and Sample Efficiency to Learn Directly on Robotic Platforms
• Sensory-Motor Control with Large Language Models via Iterative Policy Refinement

Related

• Model-Based Reinforcement Learning with SINDy
• Data-Efficient Learning for Complex and Real-Time Physical Problem Solving using Augmented Simulation
• Physics-Informed Model-Based Reinforcement Learning
• Stealing That Free Lunch: Exposing the Limits of Dyna-Style Reinforcement Learning
• A New View on Planning in Online Reinforcement Learning