A survey of deep learning techniques for autonomous driving

2019/10/17 by Sorin Grigorescu, Bogdan Trăsnea, Bogdan Trasnea +3 · 1 voice · 1,746 citations
Computer Science · Engineering · Psychology · #Advanced Neural Network Applications #Artificial intelligence #Autonomous Vehicle Technology and Safety #Computer science #Convolutional neural network #Deep learning #Engineering #Human-Automation Interaction and Safety #Human–computer interaction #Machine learning #Modular design #Motion planning #Perception #Pipeline (software) #Reinforcement learning #Robot #Robotics #cs.LG #cs.RO

paper · pdf · doi:10.1002/rob.21918

published in Journal of Field Robotics 37(3), 362-386 (Wiley) · 28 pages, 7 figures. arXiv admin note: text overlap with arXiv:1709.02435, arXiv:1610.01256 by other authors

arxiv published 2019/10/17 · openalex publication_date 2019/11/14 · arxiv created 2020/03/24 · arxiv updated 2020/03/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Abstract The last decade witnessed increasingly rapid progress in self‐driving vehicle technology, mainly backed up by advances in the area of deep learning and artificial intelligence (AI). The objective of this paper is to survey the current state‐of‐the‐art on deep learning technologies used in autonomous driving. We start by presenting AI‐based self‐driving architectures, convolutional and recurrent neural networks, as well as the deep reinforcement learning paradigm. These methodologies form a base for the surveyed driving scene perception, path planning, behavior arbitration, and motion control algorithms. We investigate both the modular perception‐planning‐action pipeline, where each module is built using deep learning methods, as well as End2End systems, which directly map sensory information to steering commands. Additionally, we tackle current challenges encountered in designing AI architectures for autonomous driving, such as their safety, training data sources, and computational hardware. The comparison presented in this survey helps gain insight into the strengths and limitations of deep learning and AI approaches for autonomous driving and assist with design choices.

Citations

• An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software
• On the Safety of Machine Learning: Cyber-Physical Systems, Decision Sciences, and Data Products
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• Adam: A Method for Stochastic Optimization
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Intelligible Models for HealthCare
• Humans and Automation: Use, Misuse, Disuse, Abuse
• Long Short-Term Memory
• Separate visual pathways for perception and action
• Gradient-based learning applied to document recognition
• Vision meets robotics: The KITTI dataset
• ImageNet Large Scale Visual Recognition Challenge
• Human-level control through deep reinforcement learning
• ImageNet classification with deep convolutional neural networks
• Representation Learning: A Review and New Perspectives
• Rapid object detection using a boosted cascade of simple features
• SSD: Single Shot MultiBox Detector
• Debugging Machine Learning Tasks
• End-to-End Tracking and Semantic Segmentation Using Recurrent Neural Networks
• End to End Learning for Self-Driving Cars
• R-FCN: Object Detection via Region-based Fully Convolutional Networks
• Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
• 1 year, 1000 km: The Oxford RobotCar dataset
• Explaining How a Deep Neural Network Trained with End-to-End Learning\n Steers a Car
• Aggressive Deep Driving: Model Predictive Control with a CNN Cost Model
• End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies
• YOLOv3: An Incremental Improvement
• Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
• Learning Dexterous In-Hand Manipulation
• PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
• Learning dexterous in-hand manipulation
• Reinforcement Learning: An Introduction
• Leveraging Deep Visual Descriptors for Hierarchical Efficient Localization
• ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
• A Multi-Modal Distributed Real-Time IoT System for Urban Traffic Control (Invited Paper)
• Computer Vision for Autonomous Vehicles: Problems, Datasets and State of the Art
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size

Cited by

• Extended Field of View Analysis for VideoGAN-based Trajectory Generation
• Drive Fast, Learn Faster: On-Board RL for High Performance Autonomous Racing
• Legilimens: Performant Video Analytics on the System-on-Chip Edge
• Scenario-based Compositional Verification of Autonomous Systems with Neural Perception
• Retina gap junctions support the robust perception by warping neural representational geometries along the visual hierarchy
• Toward Fully Autonomous Driving: AI, Challenges, Opportunities, and Needs
• Enhancing System Self-Awareness and Trust of AI: A Case Study in Trajectory Prediction and Planning
• Reachability Is NP-Complete Even for the Simplest Neural Networks
• Perception and Navigation in Autonomous Systems in the Era of Learning: A Survey
• Analysis of memory consumption by neural networks based on hyperparameters
• Collaborative Perception Datasets for Autonomous Driving: A Review
• AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systems

Discussions

• A Survey of Deep Learning Techniques for Autonomous Driving [hn, 2 points, 0 comments]

Related

• Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey
• Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
• End-to-end Autonomous Driving using Deep Learning: A Systematic Review
• Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles
• A Survey of Deep Learning Applications to Autonomous Vehicle Control