Predictive reinforcement learning based adaptive PID controller

2025/06/10 by Chaoqun Ma, Zhiyong Zhang, Ma, Chaoqun +1
Computer Science · #Adaptive Dynamic Programming Control #Advanced Technologies in Various Fields #FOS: Electrical engineering #Reinforcement Learning in Robotics #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2506.08509

openalex publication_date 2025/06/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Purpose: This study aims to address the challenges of controlling unstable and nonlinear systems by proposing an adaptive PID controller based on predictive reinforcement learning (PRL-PID), where the PRL-PID combines the advantages of both data-driven and model-driven approaches. Design/methodology/approach: A predictive reinforcement learning framework is introduced, incorporating action smooth strategy to suppress overshoot and oscillations, and a hierarchical reward function to support training. Findings: Experimental results show that the PRL-PID controller achieves superior stability and tracking accuracy in nonlinear, unstable, and strongly coupled systems, consistently outperforming existing RL-tuned PID methods while maintaining excellent robustness and adaptability across diverse operating conditions. Originality/Value: By adopting predictive learning, the proposed PRL-PID integrates system model priors into data-driven control, enhancing both the control framework's training efficiency and the controller's stability. As a result, PRL-PID provides a balanced blend of model-based and data-driven approaches, delivering robust, high-performance control.

Citations

• System Identification and Control Using Lyapunov-Based Deep Neural Networks without Persistent Excitation: A Concurrent Learning Approach
• Controlling dynamics of stochastic systems with deep reinforcement learning
• Event-Triggered Safe Bayesian Optimization on Quadcopters
• A Survey on Model-based Reinforcement Learning
• Soft Actor-Critic Algorithms and Applications
• A Tour of Reinforcement Learning: The View from Continuous Control
• Addressing Function Approximation Error in Actor-Critic Methods
• Proximal Policy Optimization Algorithms

Related

• A Model-Based Reinforcement Learning Approach for PID Design
• Stability-Preserving Automatic Tuning of PID Control with Reinforcement Learning
• Accelerate Reinforcement Learning with PID Controllers in the Pendulum Simulations
• MPC4RL -- A Software Package for Reinforcement Learning based on Model Predictive Control
• Technical Report on Reinforcement Learning Control on the Lucas-Nülle Inverted Pendulum