2024/12/22 by Fethi Bencherki, Anders Rantzer, Bencherki, Fethi +1 · 1 citation
Engineering · #Advanced Control Systems Optimization #Control Systems and Identification #FOS: Electrical engineering #FOS: Mathematics #Fault Detection and Control Systems #Optimization and Control (math.OC) #Systems and Control (eess.SY) #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2412.17012
openalex publication_date 2024/12/22 · openalex created_date 2024/12/25 · openalex updated_date 2026/07/28
An adaptive controller is proposed and analyzed for the class of infinite-horizon optimal control problems in positive linear systems presented in (Ohlin et al., 2024b). This controller is derived from the solution of a "data-driven algebraic equation" constructed using the model-free Bellman equation from Q-learning. The equation is driven by data correlation matrices that do not scale with the number of data points, enabling efficient online implementation. Consequently, a sufficient condition guaranteeing stability and robustness to unmodeled dynamics is established. The derived results also provide a quantitative characterization of the interplay between excitation level and robustness to unmodeled dynamics. The class of optimal control problems considered here is equivalent to Stochastic Shortest Path (SSP) problems, allowing for a performance comparison between the proposed adaptive policy and model-free algorithms for learning the stochastic shortest path, as demonstrated in the numerical experiment.