TubeDAgger: Reducing the Number of Expert Interventions with Stochastic Reach-Tubes

2025/10/01 by Julian Lemmel, Lemmel, Julian, Manuel Kranzl +9
Decision Sciences · Business, Management and Accounting · #Scientific Computing and Data Management #Big Data and Business Intelligence #Simulation Techniques and Applications

paper · pdf · doi:10.48550/arxiv.2510.00906

Abstract

Interactive Imitation Learning deals with training a novice policy from expert demonstrations in an online fashion. The established DAgger algorithm trains a robust novice policy by alternating between interacting with the environment and retraining of the network. Many variants thereof exist, that differ in the method of discerning whether to allow the novice to act or return control to the expert. We propose the use of stochastic reachtubes - common in verification of dynamical systems - as a novel method for estimating the necessity of expert intervention. Our approach does not require fine-tuning of decision thresholds per environment and effectively reduces the number of expert interventions, especially when compared with related approaches that make use of a doubt classification model.

Citations

• RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning
• Data-Driven Reachability Analysis of Stochastic Dynamical Systems with Conformal Inference
• SafetyNet: Safe planning for real-world self-driving vehicles using machine-learned policies
• LazyDAgger: Reducing Context Switching in Interactive Imitation Learning
• Safety Verification of Unknown Dynamical Systems via Gaussian Process\n Regression
• HG-DAgger: Interactive Imitation Learning with Human Experts
• Risk-Sensitive Generative Adversarial Imitation Learning
• EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning
• Fast Policy Learning through Imitation and Reinforcement
• Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances
• Underapproximation of Reach-Avoid Sets for Discrete-Time Stochastic Systems via Lagrangian Methods
• One-Shot Imitation Learning
• PLATO: Policy Learning using Adaptive Trajectory Optimization
• A Reduction of Imitation Learning and Structured Prediction to No-Regret\n Online Learning

Related

• CubeDAgger: Improved Robustness of Interactive Imitation Learning without Violation of Dynamic Stability
• Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
• HG-DAgger: Interactive Imitation Learning with Human Experts
• EnsembleDAgger: A Bayesian Approach to Safe Imitation Learning
• Stochastic Action Prediction for Imitation Learning