End-to-end Driving via Conditional Imitation Learning
2017/10/06 by Felipe Codevilla, Codevilla, Felipe, Matthias Müller +7 · 71 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotics (cs.RO) #cs.CV #cs.LG #cs.RO
paper · pdf · doi:10.48550/arxiv.1710.02410
Published at the International Conference on Robotics and Automation (ICRA), 2018
arxiv created 2018/03/02 · arxiv updated 2018/03/05
Abstract
Deep networks trained on demonstrations of human driving have learned to follow roads and avoid obstacles. However, driving policies trained via imitation learning cannot be controlled at test time. A vehicle trained end-to-end to imitate an expert cannot be guided to take a specific turn at an upcoming intersection. This limits the utility of such systems. We propose to condition imitation learning on high-level command input. At test time, the learned driving policy functions as a chauffeur that handles sensorimotor coordination but continues to respond to navigational commands. We evaluate different architectures for conditional imitation learning in vision-based driving. We conduct experiments in realistic three-dimensional simulations of urban driving and on a 1/5 scale robotic truck that is trained to drive in a residential area. Both systems drive based on visual input yet remain responsive to high-level navigational commands. The supplementary video can be viewed at https://youtu.be/cFtnflNe5fM
Cited by
- MUSON: A Reasoning-oriented Multimodal Dataset for Socially Compliant Navigation in Urban Environments
- Test-Time Coverage: Test-Conditioned Data Curation for Deployment-Aware Learning
- LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving
- DTCCL: Disengagement-Triggered Contrastive Continual Learning for Autonomous Bus Planners
- Distributionally Robust Imitation Learning: Layered Control Architecture for Certifiable Autonomy
- Quantifying and Bridging the Fidelity Gap: A Decisive-Feature Approach to Comparing Synthetic and Real Imagery
- Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
- ISS Policy : Scalable Diffusion Policy with Implicit Scene Supervision
- Imitation Learning for Multi-turn LM Agents via On-policy Expert Corrections
- A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approach
- COVLM-RL: Critical Object-Oriented Reasoning for Autonomous Driving Using VLM-Guided Reinforcement Learning
- VLD: Visual Language Goal Distance for Reinforcement Learning Navigation
- CoT4AD: A Vision-Language-Action Model with Explicit Chain-of-Thought Reasoning for Autonomous Driving
- Evaluating SAM2 for Video Semantic Segmentation
- LAP: Fast LAtent Diffusion Planner for Autonomous Driving
- Attacking Autonomous Driving Agents with Adversarial Machine Learning: A Holistic Evaluation with the CARLA Leaderboard
- VLA-R: Vision-Language Action Retrieval toward Open-World End-to-End Autonomous Driving
- PlanT 2.0: Exposing Biases and Structural Flaws in Closed-Loop Driving
- Follow-Me in Micro-Mobility with End-to-End Imitation Learning
- ZTRS: Zero-Imitation End-to-end Autonomous Driving with Trajectory Scoring
- VR-Drive: Viewpoint-Robust End-to-End Driving with Feed-Forward 3D Gaussian Splatting
- CGoT: A Novel Inference Mechanism for Embodied Multi-Agent Systems Using Composable Graphs of Thoughts
- Restoring Noisy Demonstration for Imitation Learning With Diffusion Models
- Align2Act: Instruction-Tuned Models for Human-Aligned Autonomous Driving
- Scalable Offline Metrics for Autonomous Driving
- Act to See, See to Act: Diffusion-Driven Perception-Action Interplay for Adaptive Policies
- SONAR: Semantic-Object Navigation with Aggregated Reasoning through a Cross-Modal Inference Paradigm
- DriveE2E: Closed-Loop Benchmark for End-to-End Autonomous Driving through Real-to-Simulation
- DynaNav: Dynamic Feature and Layer Selection for Efficient Visual Navigation
- Exploring multimodal implicit behavior learning for vehicle navigation in simulated cities
- MAP: End-to-End Autonomous Driving with Map-Assisted Planning
- SEG-Parking: Towards Safe, Efficient, and Generalizable Autonomous Parking via End-to-End Offline Reinforcement Learning
- Synthetic vs. Real Training Data for Visual Navigation
- Real-Time Obstacle Avoidance for a Mobile Robot Using CNN-Based Sensor Fusion
- Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving
- DUViN: Diffusion-Based Underwater Visual Navigation via Knowledge-Transferred Depth Features
- Generative Sequential Notification Optimization via Multi-Objective Decision Transformers
- An Initial Study of Bird's-Eye View Generation for Autonomous Vehicles using Cross-View Transformers
- Belief-Conditioned One-Step Diffusion: Real-Time Trajectory Planning with Just-Enough Sensing
- How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy
- MonoMPC: Monocular Vision Based Navigation with Learned Collision Model and Risk-Aware Model Predictive Control
- GMF-Drive: Gated Mamba Fusion with Spatial-Aware BEV Representation for End-to-End Autonomous Driving
- Safety-Aware Imitation Learning via MPC-Guided Disturbance Injection
- Adaptive Lattice-based Motion Planning
- A Concept for Efficient Scalability of Automated Driving Allowing for Technical, Legal, Cultural, and Ethical Differences
- GEMINUS: Dual-aware Global and Scene-Adaptive Mixture-of-Experts for End-to-End Autonomous Driving
- HySafe-AI: Hybrid Safety Architectural Analysis Framework for AI Systems: A Case Study
- Outcome-Guided Distillation: A Teacher-Student Framework to Advance VLM Reasoning in Autonomous Driving
- RAPiD: Reward-Guided Consistency Distillation of Diffusion Planners for Real-Time Autonomous Driving
- Imitation Learning for Satellite Attitude Control under Unknown Perturbations
- StyleDrive: Towards Driving-Style Aware Benchmarking of End-To-End Autonomous Driving
- Beyond-Expert Performance with Limited Demonstrations: Efficient Imitation Learning with Double Exploration
- ParkFormer: A Transformer-Based Parking Policy with Goal Embedding and Pedestrian-Aware Control
- SAH-Drive: A Scenario-Aware Hybrid Planner for Closed-Loop Vehicle Trajectory Generation
- S4-Driver: Scalable Self-Supervised Driving Multimodal Large Language Modelwith Spatio-Temporal Visual Representation
- CogAD: Cognitive-Hierarchy Guided End-to-End Autonomous Driving
- GaussianFusion: Gaussian-Based Multi-Sensor Fusion for End-to-End Autonomous Driving
- Echo Planning for Autonomous Driving: From Current Observations to Future Trajectories and Back
- DecoupleGS: Interactive 3D Gaussian Splatting for End-to-End Autonomous Driving Testing
- Structured Agent Distillation for Large Language Model
- Neural Multivariate Regression: Qualitative Insights from the Unconstrained Feature Model
- ACORN: Adaptive Contrastive Optimization for Safe and Robust Fine-Grained Robotic Manipulation
- An Automated Reinforcement Learning Reward Design Framework with Large Language Model for Cooperative Platoon Coordination
- Maximum Likelihood Reinforcement Learning
- Toward Fully Autonomous Driving: AI, Challenges, Opportunities, and Needs
- When Shared Rollouts Fail in Defensive Driving Evaluation: A NAVSIM Score Basis Audit
- Data Scaling Laws for End-to-End Autonomous Driving
- Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baseline
- Learning Through Retrospection: Improving Trajectory Prediction for Automated Driving with Error Feedback
- Fully Unified Motion Planning for End-to-End Autonomous Driving
- Training a Conditioned Video Game Agent on a VLM Annotated Dataset
- AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systems
Related