2025/02/21 by Austin Coursey, Marcos Quiñones-Grueiro, Coursey, Austin +3 · 1 citation
Engineering · #Advanced Control Systems Design #Control Systems and Identification #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO) #Stability and Control of Uncertain Systems
paper · pdf · doi:10.48550/arxiv.2502.15922
openalex publication_date 2025/02/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Reinforcement learning (RL) algorithms have been successfully applied to control tasks associated with unmanned aerial vehicles and robotics. In recent years, safe RL has been proposed to allow the safe execution of RL algorithms in industrial and mission-critical systems that operate in closed loops. However, if the system operating conditions change, such as when an unknown fault occurs in the system, typical safe RL algorithms are unable to adapt while retaining past knowledge. Continual reinforcement learning algorithms have been proposed to address this issue. However, the impact of continual adaptation on the system's safety is an understudied problem. In this paper, we study the intersection of safe and continual RL. First, we empirically demonstrate that a popular continual RL algorithm, online elastic weight consolidation, is unable to satisfy safety constraints in non-linear systems subject to varying operating conditions. Specifically, we study the MuJoCo HalfCheetah and Ant environments with velocity constraints and sudden joint loss non-stationarity. Then, we show that an agent trained using constrained policy optimization, a safe RL algorithm, experiences catastrophic forgetting in continual learning settings. With this in mind, we explore a simple reward-shaping method to ensure that elastic weight consolidation prioritizes remembering both safety and task performance for safety-constrained, non-linear, and non-stationary dynamical systems.