Online actor–critic algorithm to solve the continuous-time infinite horizon optimal control problem

2010/03/18 by Kyriakos G. Vamvoudakis, Frank L. Lewis · 1,654 citations
Computer Science · Engineering · Mathematics · #Adaptive Control of Nonlinear Systems #Adaptive Dynamic Programming Control #Artificial intelligence #Artificial neural network #Bellman equation #Computer science #Control (management) #Control theory (sociology) #Controller (irrigation) #Convergence (economics) #Frequency Control in Power Systems #Function (biology) #Mathematical optimization #Mathematics #Nonlinear system #Optimal control #Stability (learning theory)

paper · doi:10.1016/j.automatica.2010.02.018

published in Automatica 46(5), 878-888 (Elsevier BV)

openalex publication_date 2010/03/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/04

Citations

• Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach

Cited by

• State-Following-Kernel-Based Online Reinforcement Learning Guidance Law Against Maneuvering Target
• Temporally smoothed incremental model-based heuristic dynamic programming for command-filtered cascaded online learning flight control
• Recovery of the optimal control value function in reproducing kernel Hilbert spaces from verification conditions
• Data-based approximate policy iteration for nonlinear continuous-time optimal control design
• Closed Form HJB Solution for Continuous-Time Optimal Control of a Non-Linear Input-Affine System
• Reinforcement Learning for Fuzzy Structured Adaptive Optimal Control of Discrete-Time Nonlinear Complex Networks
• Hamilton-Jacobi Deep Q-Learning for Deterministic Continuous-Time Systems with Lipschitz Continuous Controls
• Tailoring Reproducing Kernels for Optimal Control via Policy Iteration
• Robust Actor–Critic Learning for Continuous-Time Nonlinear Systems With Unmodeled Dynamics
• Safe Output-Feedback Adaptive Optimal Control of Affine Nonlinear Systems
• Off-Policy Risk-Sensitive Reinforcement Learning Based Constrained Robust Optimal Control
• Continuous-Time Reinforcement Learning for Asset-Liability Management
• Simultaneous Identification and Optimal Tracking Control of Unknown Continuous Time Nonlinear System With Actuator Constraints Using Critic-Only Integral Reinforcement Learning
• Continuous-Time Value Iteration for Multi-Agent Reinforcement Learning
• Online inverse reinforcement learning for nonlinear systems
• HJB-based online safety-embedded critic learning for uncertain systems with self-triggered mechanism
• Q-learning for Optimal Control of Continuous-time Systems
• On the Convergence of the Policy Iteration for Infinite-Horizon Nonlinear Optimal Control Problems
• H-TD2: Hybrid Temporal Difference Learning for Adaptive Urban Taxi Dispatch
• Safe Multi-Agent Navigation via Constrained HJB-Informed Learning
• POMDPs in Continuous Time and Discrete Spaces
• Relationship Explainable Multi-objective Optimization Via Vector Value Function Based Reinforcement Learning
• Dynamic Pricing on E-commerce Platform with Deep Reinforcement Learning: A Field Experiment
• Online Adaptive Optimal Control Algorithm Based on Synchronous Integral Reinforcement Learning With Explorations
• Reinforcement learning-based trajectory tracking optimal control of unmanned surface vehicles in narrow water areas
• A Physics-Informed Learning Framework to Solve the Infinite-Horizon Optimal Control Problem
• Online Optimal Control with Linear Dynamics and Predictions: Algorithms and Regret Analysis
• Continuous-Time Model-Based Reinforcement Learning
• Online Approximate Optimal Station Keeping of an Autonomous Underwater Vehicle
• Verification for Machine Learning, Autonomy, and Neural Networks Survey
• On performance bound estimation in NMPC with time-varying terminal cost
• Critic-Only Integral Reinforcement Learning Driven by Variable Gain Gradient Descent for Optimal Tracking Control
• Online optimal adaptive control of a catalytic cracking reactor via integral reinforcement learning
• Data-Driven Optimal Assistance Control of a Lower Limb Exoskeleton for Hemiplegic Patients. [europepmc]
• Survey of Image Processing Techniques for Brain Pathology Diagnosis: Challenges and Opportunities. [europepmc]
• Application of reinforcement learning for effective vaccination strategies of coronavirus disease 2019 (COVID-19). [europepmc]
• Adaptive Critic Learning-Based Robust Control of Systems with Uncertain Dynamics. [europepmc]
• Robust Trajectory Tracking Control for Continuous-Time Nonlinear Systems with State Constraints and Uncertain Disturbances. [europepmc]
• Critic Learning-Based Safe Optimal Control for Nonlinear Systems with Asymmetric Input Constraints and Unmatched Disturbances. [europepmc]
• Optimal Robust Control of Nonlinear Systems with Unknown Dynamics via NN Learning with Relaxed Excitation. [europepmc]
• Predefined-Time Sliding Mode Control of Robotic Manipulators via Artificial Delay Feedback and Reinforcement Learning. [europepmc]
• Adaptive Predefined-Time Tracking Control for Robotic Manipulator Based on Actor-Critic Reinforcement Learning [europepmc]