Exploring the Limitations of Behavior Cloning for Autonomous Driving
2019/04/18 by Felipe Codevilla, Codevilla, Felipe, Eder Santana +5 · 48 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #cs.AI #cs.CV
paper · pdf · doi:10.48550/arxiv.1904.08980
arxiv created 2019/04/18 · arxiv updated 2019/04/22
Abstract
Driving requires reacting to a wide variety of complex environment conditions and agent behaviors. Explicitly modeling each possible scenario is unrealistic. In contrast, imitation learning can, in theory, leverage data from large fleets of human-driven cars. Behavior cloning in particular has been successfully used to learn simple visuomotor policies end-to-end, but scaling to the full spectrum of driving behaviors remains an unsolved problem. In this paper, we propose a new benchmark to experimentally investigate the scalability and limitations of behavior cloning. We show that behavior cloning leads to state-of-the-art results, including in unseen environments, executing complex lateral and longitudinal maneuvers without these reactions being explicitly programmed. However, we confirm well-known limitations (due to dataset bias and overfitting), new generalization issues (due to dynamic objects and the lack of a causal model), and training instability requiring further research before behavior cloning can graduate to real-world driving. The code of the studied behavior cloning approaches can be found at https://github.com/felipecode/coiltraine .
Cited by
- MUSON: A Reasoning-oriented Multimodal Dataset for Socially Compliant Navigation in Urban Environments
- InDRiVE: Reward-Free World-Model Pretraining for Autonomous Driving via Latent Disagreement
- TakeAD: Preference-based Post-optimization for End-to-end Autonomous Driving with Expert Takeover Data
- Towards Closing the Domain Gap with Event Cameras
- Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
- Sequence of Expert: Boosting Imitation Planners for Autonomous Driving through Temporal Alternation
- Advancing Autonomous Driving System Testing: Demands, Challenges, and Future Directions
- RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policies
- Consistent inverse optimal control for discrete-time nonlinear stochastic systems
- Attacking Autonomous Driving Agents with Adversarial Machine Learning: A Holistic Evaluation with the CARLA Leaderboard
- Continuous Vision-Language-Action Co-Learning with Semantic-Physical Alignment for Behavioral Cloning
- VLA-R: Vision-Language Action Retrieval toward Open-World End-to-End Autonomous Driving
- PlanT 2.0: Exposing Biases and Structural Flaws in Closed-Loop Driving
- AdaDrive: Self-Adaptive Slow-Fast System for Language-Grounded Autonomous Driving
- VLDrive: Vision-Augmented Lightweight MLLMs for Efficient Language-grounded Autonomous Driving
- RoCA: Robust Cross-Domain End-to-End Autonomous Driving
- Dino-Diffusion Modular Designs Bridge the Cross-Domain Gap in Autonomous Parking
- HYPE: Hybrid Planning with Ego Proposal-Conditioned Predictions
- FlowDrive: moderated flow matching with data balancing for trajectory planning
- Exploring multimodal implicit behavior learning for vehicle navigation in simulated cities
- TreeIRL: Safe Urban Driving with Tree Search and Inverse Reinforcement Learning
- Group Effect Enhanced Generative Adversarial Imitation Learning for Individual Travel Behavior Modeling under Incentives
- Mini Autonomous Car Driving based on 3D Convolutional Neural Networks
- Interpretable Decision-Making for End-to-End Autonomous Driving
- ReconDreamer-RL: Enhancing Reinforcement Learning via Diffusion-based Scene Reconstruction
- From Imitation to Optimization: A Comparative Study of Offline Learning for Autonomous Driving
- From Seeing to Experiencing: Scaling Navigation Foundation Models with Reinforcement Learning
- A Concept for Efficient Scalability of Automated Driving Allowing for Technical, Legal, Cultural, and Ethical Differences
- GEMINUS: Dual-aware Global and Scene-Adaptive Mixture-of-Experts for End-to-End Autonomous Driving
- VLAD: A VLM-Augmented Autonomous Driving Framework with Hierarchical Planning and Interpretable Decision Process
- Robust Behavior Cloning Via Global Lipschitz Regularization
- Learning dissection trajectories from expert surgical videos via imitation learning with equivariant diffusion
- Confidence-Guided Human-AI Collaboration: Reinforcement Learning with Distributional Proxy Value Propagation for Autonomous Driving
- SAH-Drive: A Scenario-Aware Hybrid Planner for Closed-Loop Vehicle Trajectory Generation
- Impromptu VLA: Open Weights and Open Data for Driving Vision-Language-Action Models
- HMAD: Advancing E2E Driving with Anchored Offset Proposals and Simulation-Supervised Multi-target Scoring
- GaussianFusion: Gaussian-Based Multi-Sensor Fusion for End-to-End Autonomous Driving
- DiffE2E: Rethinking End-to-End Driving with a Hybrid Action Diffusion and Supervised Policy
- Echo Planning for Autonomous Driving: From Current Observations to Future Trajectories and Back
- HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving
- Saliency-Aware Quantized Imitation Learning for Efficient Robotic Control
- iPad: Iterative Proposal-centric End-to-End Autonomous Driving
- Generative AI for Autonomous Driving: A Review
- Scaling Self-Play for End-to-End Driving
- Toward Fully Autonomous Driving: AI, Challenges, Opportunities, and Needs
- Learning Through Retrospection: Improving Trajectory Prediction for Automated Driving with Error Feedback
- Fully Unified Motion Planning for End-to-End Autonomous Driving
- An inverse mixed-integer optimization framework for learning interpretable models of expert decision making
Related