An efficient data-based off-policy Q-learning algorithm for optimal output feedback control of linear systems

2023/12/06 by Mohammad Alsalti, Alsalti, Mohammad, Victor G. Lopez +3
Engineering · #Advanced Control Systems Optimization #FOS: Electrical engineering #Fuel Cells and Related Materials #Iterative Learning Control Systems #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2312.03451

openalex publication_date 2023/12/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/03

Abstract

In this paper, we present a Q-learning algorithm to solve the optimal output regulation problem for discrete-time LTI systems. This off-policy algorithm only relies on using persistently exciting input-output data, measured offline. No model knowledge or state measurements are needed and the obtained optimal policy only uses past input-output information. Moreover, our formulation of the proposed algorithm renders it computationally efficient. We provide conditions that guarantee the convergence of the algorithm to the optimal solution. Finally, the performance of our method is compared to existing algorithms in the literature.

Related

• Robust Output Regulation and Reinforcement Learning-based Output Tracking Design for Unknown Linear Discrete-Time Systems
• Optimal Output Feedback Learning Control for Discrete-Time Linear Quadratic Regulation
• Data-Driven Control of Unknown Systems: A Linear Programming Approach
• An Efficient Off-Policy Reinforcement Learning Algorithm for the Continuous-Time LQR Problem
• Q-learning for Optimal Control of Continuous-time Systems