Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning

2021/08/13 by Lukas Brunke, Melissa Greeff, Brunke, Lukas +11 · 54 citations
Computer Science · Engineering · Health Professions · #FOS: Computer and information sciences #FOS: Electrical engineering #Fault Detection and Control Systems #Machine Learning (cs.LG) #Occupational Health and Safety Research #Robotics (cs.RO) #Software Reliability and Analysis Research #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · pdf · doi:10.48550/arxiv.2108.06266

openalex publication_date 2021/08/13 · openalex created_date 2022/07/25 · openalex updated_date 2026/07/28

Abstract

The last half-decade has seen a steep rise in the number of contributions on safe learning methods for real-world robotic deployments from both the control and reinforcement learning communities. This article provides a concise but holistic review of the recent advances made in using machine learning to achieve safe decision making under uncertainties, with a focus on unifying the language and frameworks used in control theory and reinforcement learning research. Our review includes: learning-based control approaches that safely improve performance by learning the uncertain dynamics, reinforcement learning approaches that encourage safety or robustness, and methods that can formally certify the safety of a learned control policy. As data- and learning-based robot control methods continue to gain traction, researchers must understand when and how to best leverage them in real-world scenarios where safety is imperative, such as when operating in close proximity to humans. We highlight some of the open challenges that will drive the field of robot learning in the coming years, and emphasize the need for realistic physics-based benchmarks to facilitate fair comparisons between control and reinforcement learning approaches.

Cited by

• Lies We Can Trust: Quantifying Action Uncertainty with Inaccurate Stochastic Dynamics through Conformalized Nonholonomic Lie Groups
• VLSA: Vision-Language-Action Models with Plug-and-Play Safety Constraint Layer
• ContactRL: Safe Reinforcement Learning based Motion Planning for Contact based Human Robot Collaboration
• Probabilistic Safety under Arbitrary Disturbance Distributions using Piecewise-Affine Control Barrier Functions
• SOMBRL: Scalable and Optimistic Model-Based RL
• LEARN: Learning End-to-End Aerial Resource-Constrained Multi-Robot Navigation
• Platform-Agnostic Reinforcement Learning Framework for Safe Exploration of Cluttered Environments with Graph Attention
• Physics-Informed Machine Learning for Characterizing System Stability
• From Demonstrations to Safe Deployment: Path-Consistent Safety Filtering for Diffusion Policies
• Learning Dynamics from Input-Output Data with Hamiltonian Gaussian Processes
• Exchange Policy Optimization Algorithm for Semi-Infinite Safe Reinforcement Learning
• Safe and time-efficient exploration in Reinforcement Learning-based control of a vehicle thermal systems
• Control Synthesis with Reinforcement Learning: A Modeling Perspective
• K-DAREK: Distance Aware Error for Kurkova Kolmogorov Networks
• Safety Assessment in Reinforcement Learning via Model Predictive Control
• MADR: MPC-guided Adversarial DeepReach
• Few-Shot Demonstration-Driven Task Coordination and Trajectory Execution for Multi-Robot Systems
• STEMS: Spatial-Temporal Enhanced Safe Multi-Agent Coordination for Building Energy Management
• A Primer on SO(3) Action Representations in Deep Reinforcement Learning
• Failure Prediction at Runtime for Generative Robot Policies
• ECLipsE-Gen-Local: Efficient Compositional Local Lipschitz Estimates for Deep Neural Networks
• Off-Policy Reinforcement Learning with Anytime Safety Guarantees via Robust Safe Gradient Flow
• Memory-Driven Self-Improvement for Decision Making with Large Language Models
• CaRe-BN: Precise Moving Statistics for Stabilizing Spiking Neural Networks in Reinforcement Learning
• Goal-Guided Efficient Exploration via Large Language Model in Reinforcement Learning
• From Space to Time: Enabling Adaptive Safety with Learned Value Functions via Disturbance Recasting
• Security of World-Model-Based Embodied AI: A Lifecycle of Threats, Defenses, and Evaluation
• SPiDR: A Simple Approach for Zero-Shot Safety in Sim-to-Real Transfer
• Guaranteed Robust Nonlinear MPC via Disturbance Feedback
• Distributionally Robust Safe Motion Planning with Contextual Information
• Zero-Shot Transferable Solution Method for Parametric Optimal Control Problems
• EigenSafe: A Spectral Framework for Learning-Based Stochastic Safety Filtering
• RL-augmented Adaptive Model Predictive Control for Bipedal Locomotion over Challenging Terrain
• Safe Guaranteed Dynamics Exploration with Probabilistic Models
• Uncertainty-Based Smooth Policy Regularisation for Reinforcement Learning with Few Demonstrations
• A predictive modular approach to constraint satisfaction under uncertainty - with application to glycosylation in continuous monoclonal antibody biosimilar production
• When Large Language Models Meet UAV Projects: An Empirical Study from Developers' Perspective
• Continuous-Time Value Iteration for Multi-Agent Reinforcement Learning
• Off Policy Lyapunov Stability in Reinforcement Learning
• Reinforcement Learning-based Control via Y-wise Affine Neural Networks (YANNs)
• Reservoir Predictive Path Integral Control for Unknown Nonlinear Dynamics
• Enhancing Reliability in LLM-Integrated Robotic Systems: A Unified Approach to Security and Safety
• End-to-End Low-Level Neural Control of an Industrial-Grade 6D Magnetic Levitation System
• Bridging Minds and Machines: Toward an Integration of AI and Cognitive Science
• Towards safe control parameter tuning in distributed multi-agent systems
• SIGN: Safety-Aware Image-Goal Navigation for Autonomous Drones via Reinforcement Learning
• Interpretable Robot Control via Structured Behavior Trees and Large Language Models
• A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions
• Towards Safe Imitation Learning via Potential Field-Guided Flow Matching
• End-to-End Humanoid Robot Safe and Comfortable Locomotion Policy
• Beyond Constant Parameters: Hyper Prediction Models and HyperMPC
• Improving Drone Racing Performance Through Iterative Learning MPC
• Physics-Informed EvolveGCN: Satellite Prediction for Multi Agent Systems
• A roadmap for AI in robotics

Related

• Survey on safe robot control via learning
• safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Robotics
• Safe Learning for Contact-Rich Robot Tasks: A Survey from Classical Learning-Based Methods to Safe Foundation Models
• Safety Correction from Baseline: Towards the Risk-aware Policy in Robotics via Dual-agent Reinforcement Learning
• A Review of Safe Reinforcement Learning: Methods, Theory and Applications