2022/07/27 by Masaki Waga, Waga, Masaki, Ezequiel Castellano +9 · 4 citations
Computer Science · #FOS: Computer and information sciences #Formal Languages and Automata Theory (cs.FL) #Machine Learning (cs.LG) #Machine Learning and Algorithms #Reinforcement Learning in Robotics #Software Reliability and Analysis Research
paper · pdf · doi:10.48550/arxiv.2207.13446
openalex publication_date 2022/07/27 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
It is challenging to use reinforcement learning (RL) in cyber-physical systems due to the lack of safety guarantees during learning. Although there have been various proposals to reduce undesired behaviors during learning, most of these techniques require prior system knowledge, and their applicability is limited. This paper aims to reduce undesired behaviors during learning without requiring any prior system knowledge. We propose dynamic shielding: an extension of a model-based safe RL technique called shielding using automata learning. The dynamic shielding technique constructs an approximate system model in parallel with RL using a variant of the RPNI algorithm and suppresses undesired explorations due to the shield constructed from the learned model. Through this combination, potentially unsafe actions can be foreseen before the agent experiences them. Experiments show that our dynamic shield significantly decreases the number of undesired events during training.