Guided by Guardrails: Control Barrier Functions as Safety Instructors for Robotic Learning

2025/05/24 by Maeva Guerrier, Guerrier, Maeva, Karthik Soma +5
Decision Sciences · Health Professions · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Occupational Health and Safety Research #Risk and Safety Analysis #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2505.18858

openalex publication_date 2025/05/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Safety stands as the primary obstacle preventing the widespread adoption of learning-based robotic systems in our daily lives. While reinforcement learning (RL) shows promise as an effective robot learning paradigm, conventional RL frameworks often model safety by using single scalar negative rewards with immediate episode termination, failing to capture the temporal consequences of unsafe actions (e.g., sustained collision damage). In this work, we introduce a novel approach that simulates these temporal effects by applying continuous negative rewards without episode termination. Our experiments reveal that standard RL methods struggle with this model, as the accumulated negative values in unsafe zones create learning barriers. To address this challenge, we demonstrate how Control Barrier Functions (CBFs), with their proven safety guarantees, effectively help robots avoid catastrophic regions while enhancing learning outcomes. We present three CBF-based approaches, each integrating traditional RL methods with Control Barrier Functions, guiding the agent to learn safe behavior. Our empirical analysis, conducted in both simulated environments and real-world settings using a four-wheel differential drive robot, explores the possibilities of employing these approaches for safe robotic learning.

Citations

• Can a Bayesian Oracle Prevent Harm from an Agent?
• Intentionally-underestimated Value Function at Terminal State for Temporal-difference Learning with Mis-designed Reward
• Champion-level drone racing using deep reinforcement learning
• Sim-to-Lab-to-Real: Safe reinforcement learning with shielding and generalization guarantees
• A Simple Reward-free Approach to Constrained Reinforcement Learning
• Exploration-Exploitation in Constrained MDPs
• Learning for Safety-Critical Control with Control Barrier Functions
• Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamics
• Control Barrier Functions: Theory and Applications
• End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks
• A predictive safety filter for learning-based control of constrained\n nonlinear dynamical systems
• Constrained Exploration and Recovery from Experience Shaping
• Reward Constrained Policy Optimization
• Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor
• Safe Reinforcement Learning via Shielding
• Constrained Policy Optimization
• Gymnasium: A Standard Interface for Reinforcement Learning Environments

Related

• Learning Control Barrier Functions and their application in Reinforcement Learning: A Survey
• Reducing Safety Interventions in Provably Safe Reinforcement Learning
• CBF-RL: Safety Filtering Reinforcement Learning in Training with Control Barrier Functions
• A Dynamic Safety Shield for Safe and Efficient Reinforcement Learning of Navigation Tasks
• Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning