End to End Learning for Self-Driving Cars
2016/04/25 by Mariusz Bojarski, Bojarski, Mariusz, Davide Testa +25 · 3 voices · 545 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Autonomous Vehicle Technology and Safety #cs.CV #cs.LG #cs.NE
paper · pdf · doi:10.48550/arxiv.1604.07316
arxiv created 2016/04/25 · arxiv updated 2016/04/26
Abstract
We trained a convolutional neural network (CNN) to map raw pixels from a single front-facing camera directly to steering commands. This end-to-end approach proved surprisingly powerful. With minimum training data from humans the system learns to drive in traffic on local roads with or without lane markings and on highways. It also operates in areas with unclear visual guidance such as in parking lots and on unpaved roads. The system automatically learns internal representations of the necessary processing steps such as detecting useful road features with only the human steering angle as the training signal. We never explicitly trained it to detect, for example, the outline of roads. Compared to explicit decomposition of the problem, such as lane marking detection, path planning, and control, our end-to-end system optimizes all processing steps simultaneously. We argue that this will eventually lead to better performance and smaller systems. Better performance will result because the internal components self-optimize to maximize overall system performance, instead of optimizing human-selected intermediate criteria, e.g., lane detection. Such criteria understandably are selected for ease of human interpretation which doesn't automatically guarantee maximum system performance. Smaller networks are possible because the system learns to solve the problem with the minimal number of processing steps. We used an NVIDIA DevBox and Torch 7 for training and an NVIDIA DRIVE(TM) PX self-driving car computer also running Torch 7 for determining where to drive. The system operates at 30 frames per second (FPS).
Citations
Cited by
- How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learned
- Computing Systems for Autonomous Driving: State-of-the-Art and Challenges
- A Survey on Autonomous Vehicle Control in the Era of Mixed-Autonomy: From Physics-Based to AI-Guided Driving Policy Learning
- Imitation Learning Approach for AI Driving Olympics Trained on Real-world and Simulation Data Simultaneously
- A Methodological Review of Visual Road Recognition Procedures for Autonomous Driving Applications
- RTGPU: Real-Time GPU Scheduling of Hard Deadline Parallel Tasks with Fine-Grain Utilization
- What data do we need for training an AV motion planner?
- MOJITO: Modal Joint Learning for Unified End-to-End Autonomous Driving
- Budgeted Training: Rethinking Deep Neural Network Training Under Resource Constraints
- Black-box Adversarial Sample Generation Based on Differential Evolution
- Guided Policy Search Model-based Reinforcement Learning for Urban Autonomous Driving
- Exact and Asymptotically Complete Robust Verifications of Neural Networks via Ising Solvers
- SOON: Scenario Oriented Object Navigation with Graph-based Exploration
- Microscopic Traffic Simulation by Cooperative Multi-agent Deep Reinforcement Learning
- LEAD: Minimizing Learner-Expert Asymmetry in End-to-End Driving
- Goal-conditioned Imitation Learning
- Learning by Cheating
- DeepFault: Fault Localization for Deep Neural Networks
- DTCCL: Disengagement-Triggered Contrastive Continual Learning for Autonomous Bus Planners
- QoS-Aware Load Balancing in the Computing Continuum via Multi-Player Bandits
- Misbehavior Forecasting for Focused Autonomous Driving Systems Testing
- Offline Behavioral Data Selection
- Posterior Behavioral Cloning: Pretraining BC Policies for Efficient RL Finetuning
- Towards Closing the Domain Gap with Event Cameras
- Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future
- Imitation Learning for Multi-turn LM Agents via On-policy Expert Corrections
- Sequence of Expert: Boosting Imitation Planners for Autonomous Driving through Temporal Alternation
- DeepSmartFuzzer: Reward Guided Test Generation For Deep Learning
- High Order Control Lyapunov Function - Control Barrier Function - Quadratic Programming Based Autonomous Driving Controller for Bicyclist Safety
- Evolutionary Synthesis of Deep Neural Networks via Synaptic Cluster-driven Genetic Encoding
- State-Conditional Adversarial Learning: An Off-Policy Visual Domain Transfer Method for End-to-End Imitation Learning
- A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approach
- FutureX: Enhance End-to-End Autonomous Driving via Latent Chain-of-Thought World Model
- Neuronal Attention Circuit (NAC) for Representation Learning
- A2D2: Audi Autonomous Driving Dataset
- TrajMoE: Scene-Adaptive Trajectory Planning with Mixture of Experts and Reinforcement Learning
- Spatial Retrieval Augmented Autonomous Driving
- NEARL: Non-Explicit Action Reinforcement Learning for Robotic Control
- CoT4AD: A Vision-Language-Action Model with Explicit Chain-of-Thought Reasoning for Autonomous Driving
- RoaD: Rollouts as Demonstrations for Closed-Loop Supervised Fine-Tuning of Autonomous Driving Policies
- All-Weather Object Recognition Using Radar and Infrared Sensing
- Provably Minimally-Distorted Adversarial Examples
- A Review of Formal Methods applied to Machine Learning
- Dataset Poisoning Attacks on Behavioral Cloning Policies
- Improving the Generalization of End-to-End Driving through Procedural Generation
- Modeling and Optimizing Performance Bottlenecks for Neuromorphic Accelerators
- WPT: World-to-Policy Transfer via Online World Model Distillation
- Visual Imitation Made Easy
- Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations
- Neural Network Branch-and-Bound for Neural Network Verification
- Why Calibration Error is Wrong Given Model Uncertainty: Using Posterior Predictive Checks with Deep Learning
- CBMA: Improving conformal prediction through Bayesian model averaging
- DiffRefiner: Coarse to Fine Trajectory Planning via Diffusion Refinement with Semantic Interaction for End to End Autonomous Driving
- Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems
- Uncertainty-Aware Data Aggregation for Deep Imitation Learning
- Learning Human-Like RL Agents Through Trajectory Optimization With Action Quantization
- HMC: Learning Heterogeneous Meta-Control for Contact-Rich Loco-Manipulation
- PAC Prediction Sets Under Covariate Shift
- Roofline Model for UAVs:A Bottleneck Analysis Tool for Designing Compute Systems for Autonomous Drones
- Preventing Imitation Learning with Adversarial Policy Ensembles
- DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars
- Multi-lane Cruising Using Hierarchical Planning and Reinforcement Learning
- Solving an elastic inverse problem using Convolutional Neural Networks
- Learning Accurate, Comfortable and Human-like Driving
- Learning Awareness Models
- Blockchain-Powered Collaboration in Heterogeneous Swarms of Robots
- Extremal Contours: Gradient-driven contours for compact visual attribution
- VLA-R: Vision-Language Action Retrieval toward Open-World End-to-End Autonomous Driving
- Multi-task Learning with Attention for End-to-end Autonomous Driving
- Cross-domain Imitation from Observations
- Model-Predictive Policy Learning with Uncertainty Regularization for Driving in Dense Traffic
- A Hierarchical Architecture for Sequential Decision-Making in Autonomous Driving using Deep Reinforcement Learning
- Proof Minimization in Neural Network Verification
- ROBY: Evaluating the Robustness of a Deep Model by its Decision Boundaries
- Examining the causal structures of deep neural networks using information theory
- UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers
- Action-Based Representation Learning for Autonomous Driving
- Saliency-Guided Domain Adaptation for Left-Hand Driving in Autonomous Steering
- SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning
- Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tail
- Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translation
- Maximum Resilience of Artificial Neural Networks
- Sidekick Policy Learning for Active Visual Exploration
- Novelty Detection via Network Saliency in Visual-based Deep Learning
- Policy Smoothing for Provably Robust Reinforcement Learning
- Avoidance of Manual Labeling in Robotic Autonomous Navigation Through Multi-Sensory Semi-Supervised Learning
- Imitation Learning for End to End Vehicle Longitudinal Control with Forward Camera
- The Hidden Vulnerability of Distributed Learning in Byzantium
- Feedback in Imitation Learning: The Three Regimes of Covariate Shift
- Continual Learning for Robotics: Definition, Framework, Learning Strategies, Opportunities and Challenges
- Toward Simulating Environments in Reinforcement Learning Based Recommendations
- CopyCAT: Taking Control of Neural Policies with Constant Attacks
- Compositional ADAM: An Adaptive Compositional Solver
- Instance-Aware Predictive Navigation in Multi-Agent Environments
- PyTorch Distributed: Experiences on Accelerating Data Parallel Training
- On Calibration of Modern Neural Networks
- Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
- On Value Discrepancy of Imitation Learning
- PatchAttack: A Black-box Texture-based Attack with Reinforcement Learning
- Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning
- Soft Calibration Objectives for Neural Networks
- Probabilistic Verification of Neural Networks Against Group Fairness
- Learning End-to-end Multimodal Sensor Policies for Autonomous Navigation
- Automating Vehicles by Deep Reinforcement Learning using Task Separation with Hill Climbing
- One-Shot Visual Imitation Learning via Meta-Learning
- Perceptual Attention-based Predictive Control
- Demystifying Parallel and Distributed Deep Learning: An In-Depth Concurrency Analysis
- Path Integral Networks: End-to-End Differentiable Optimal Control
- Neural Ensemble Search for Uncertainty Estimation and Dataset Shift
- A Survey of FPGA-Based Robotic Computing
- TrafficSim: Learning to Simulate Realistic Multi-Agent Behaviors
- SOCRATES: Towards a Unified Platform for Neural Network Analysis
- Context-Aware Image Denoising with Auto-Threshold Canny Edge Detection to Suppress Adversarial Perturbation
- Social Attention for Autonomous Decision-Making in Dense Traffic
- Explaining How a Deep Neural Network Trained with End-to-End Learning\n Steers a Car
- Optimality of Sub-network Laplace Approximations: New Results and Methods
- Dynamic Adversarial Patch for Evading Object Detection Models
- Toward Intelligent Sensing: Intermediate Deep Feature Compression
- Attacking Vision-based Perception in End-to-End Autonomous Driving Models
- AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles
- Meta-Adversarial Inverse Reinforcement Learning for Decision-making Tasks
- Fighting Copycat Agents in Behavioral Cloning from Observation Histories
- Vision-Language Navigation with Self-Supervised Auxiliary Reasoning Tasks
- Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment
- Wolf in Sheep's Clothing - The Downscaling Attack Against Deep Learning Applications
- Systematic Testing of Convolutional Neural Networks for Autonomous Driving
- Detection Defense Against Adversarial Attacks with Saliency Map
- Reinforcement Learning from Imperfect Demonstrations
- Countering Adversarial Images using Input Transformations
- CGoT: A Novel Inference Mechanism for Embodied Multi-Agent Systems Using Composable Graphs of Thoughts
- Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
- Imitating Driver Behavior with Generative Adversarial Networks
- Efficient and Robust LiDAR-Based End-to-End Navigation
- Interpretable End-to-end Urban Autonomous Driving with Latent Deep Reinforcement Learning
- Dino-Diffusion Modular Designs Bridge the Cross-Domain Gap in Autonomous Parking
- Advances in Inference and Representation for Simultaneous Localization and Mapping
- AutoMT: A Multi-Agent LLM Framework for Automated Metamorphic Testing of Autonomous Driving Systems
- Consistent Zero-Shot Imitation with Contrastive Goal Inference
- Deep Learning-Based Autonomous Driving Systems: A Survey of Attacks and Defenses
- Adaptive Individual Uncertainty under Out-Of-Distribution Shift with Expert-Routed Conformal Prediction
- Efficient Learning of Distributed Linear-Quadratic Controllers
- Diversity regularization in deep ensembles
- Multivariate Deep Evidential Regression
- Are Labels Required for Improving Adversarial Robustness?
- Improving Robustness using Generated Data
- An Explorative Study on Distributed Computing Techniques in Training and Inference of Large Language Models
- Neutral Agent-based Adversarial Policy Learning against Deep Reinforcement Learning in Multi-party Open Systems
- Flow Matching-Based Autonomous Driving Planning with Advanced Interactive Behavior Modeling
- Align2Act: Instruction-Tuned Models for Human-Aligned Autonomous Driving
- Learning to Drive using Inverse Reinforcement Learning and Deep Q-Networks
- Scalable Offline Metrics for Autonomous Driving
- On Scalable Testing of Samplers
- Yaw-Guided Imitation Learning for Autonomous Driving in Urban Environments
- On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstration
- Learning-based model predictive control with moving horizon state estimation for autonomous racing
- Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AI
- A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation
- Exploring Uncertainty in Deep Learning for Construction of Prediction Intervals
- Learning-Based Testing for Deep Learning: Enhancing Model Robustness with Adversarial Input Prioritization
- End-to-end Multi-Modal Multi-Task Vehicle Control for Self-Driving Cars with Visual Perception
- A New Approach for Resource Scheduling with Deep Reinforcement Learning
- Learning hierarchical behavior and motion planning for autonomous driving
- DeepRacing: Parameterized Trajectories for Autonomous Racing
- Combinatorial Testing for Deep Learning Systems
- A Multi-Modality Evaluation of the Reality Gap in Autonomous Driving Systems
- Parameterized Hardness of Zonotope Containment and Neural Network Verification
- DeepGuard: A Framework for Safeguarding Autonomous Driving Systems from Inconsistent Behavior
- Query-Efficient Imitation Learning for End-to-End Autonomous Driving
- Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectory
- Approaching Neural Network Uncertainty Realism
- Affordance-based Reinforcement Learning for Urban Driving
- Shaping Reward Signals in Reinforcement Learning Using Constraint Programming
- Moral Dilemmas for Moral Machines
- A First Look at Mobile Intelligence: Architecture, Experimentation and Challenges
- Real-time End-to-End Federated Learning: An Automotive Case Study
- Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap
- AutoSoC: Automating Algorithm-SOC Co-design for Aerial Robots
- Automated Analysis of Femoral Artery Calcification Using Machine Learning Techniques
- Time-Dependent Hybrid-State A* and Optimal Control for Autonomous Vehicles in Arbitrary and Dynamic Environment
- Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free
- A Survey of Autonomous Driving: Common Practices and Emerging Technologies
- Model-free Deep Reinforcement Learning for Urban Autonomous Driving
- Find Your Own Way: Weakly-Supervised Segmentation of Path Proposals for Urban Autonomy
- VisualBackProp: efficient visualization of CNNs
- Hierarchical Multi-task Deep Neural Network Architecture for End-to-End Driving
- Neural Rate Control for Video Encoding using Imitation Learning
- How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective
- Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving
- iFinder: Structured Zero-Shot Vision-Based LLM Grounding for Dash-Cam Video Reasoning
- nn-dependability-kit: Engineering Neural Networks for Safety-Critical Autonomous Driving Systems
- TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
- Learning to Steer by Mimicking Features from Heterogeneous Auxiliary Networks
- Reinforcement Learning Agent for a 2D Shooter Game
- Exploring multimodal implicit behavior learning for vehicle navigation in simulated cities
- Learning a CNN-based End-to-End Controller for a Formula SAE Racecar
- MAP: End-to-End Autonomous Driving with Map-Assisted Planning
- TreeIRL: Safe Urban Driving with Tree Search and Inverse Reinforcement Learning
- DARTS: Deceiving Autonomous Cars with Toxic Signs
- Grounding Human-to-Vehicle Advice for Self-driving Vehicles
- Unifying Variational Inference and PAC-Bayes for Supervised Learning that Scales
- Uncovering the Limits of Adversarial Training against Norm-Bounded Adversarial Examples
- Autonomous Driving with Deep Learning: A Survey of State-of-Art Technologies
- Learning to Navigate Unseen Environments: Back Translation with Environmental Dropout
- DeepBillboard: Systematic Physical-World Testing of Autonomous Driving Systems
- Learning Self-Imitating Diverse Policies
- Self-Augmented Robot Trajectory: Efficient Imitation Learning via Safe Self-augmentation with Demonstrator-annotated Precision
- Deep Model Predictive Control with Stability Guarantees
- DeepGini: Prioritizing Massive Tests to Enhance the Robustness of Deep Neural Networks
- RISC-NN: Use RISC, NOT CISC as Neural Network Hardware Infrastructure
- A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles
- Revisiting the Calibration of Modern Neural Networks
- SQIL: Imitation Learning via Reinforcement Learning with Sparse Rewards
- Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving
- Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving
- Simpler Certified Radius Maximization by Propagating Covariances
- TopoMap: A Feature-based Semantic Discriminator of the Topographical Regions in the Test Input Space
- DUViN: Diffusion-Based Underwater Visual Navigation via Knowledge-Transferred Depth Features
- Autonomous Learning From Success and Failure: Goal-Conditioned Supervised Learning with Negative Feedback
- Safer End-to-End Autonomous Driving via Conditional Imitation Learning and Command Augmentation
- A Polynomial-Based Approach for Architectural Design and Learning with\n Deep Neural Networks
- PIVEN: A Deep Neural Network for Prediction Intervals with Specific Value Prediction
- TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systems
- End-to-End Race Driving with Deep Reinforcement Learning
- Flexible Deep Neural Network Processing
- ADAPS: Autonomous Driving Via Principled Simulations
- End-to-End Low-Level Neural Control of an Industrial-Grade 6D Magnetic Levitation System
- Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples
- Clipping free attacks against artificial neural networks
- Deep k-Nearest Neighbors: Towards Confident, Interpretable and Robust Deep Learning
- Causal Learning and Explanation of Deep Neural Networks via Autoencoded Activations
- In Defense of Classical Image Processing: Fast Depth Completion on the CPU
- On Offline Evaluation of Vision-based Driving Models
- Automated Test Validators for Flaky Cyber-Physical System Simulators: Approach and Evaluation
- Improving Calibration for Long-Tailed Recognition
- A Survey on Automated Driving System Testing: Landscapes and Trends
- AI Research Considerations for Human Existential Safety (ARCHES)
- Mini-Batch Robustness Verification of Deep Neural Networks
- Mining the Long Tail: A Comparative Study of Data-Centric Criticality Metrics for Robust Offline Reinforcement Learning in Autonomous Motion Planning
- Decentralized Vision-Based Autonomous Aerial Wildlife Monitoring
- Vision-Based Autonomous Car Racing Using Deep Imitative Reinforcement Learning
- Towards Unified Probabilistic Verification and Validation of Vision-Based Autonomy
- DRIBO: Robust Deep Reinforcement Learning via Multi-View Information Bottleneck
- Semi-Infinite Programming for Collision-Avoidance in Optimal and Model Predictive Control
- Adma: A Flexible Loss Function for Neural Networks
- Automated Verification of Neural Networks: Advances, Challenges and Perspectives
- Improving Robustness of Learning-based Autonomous Steering Using Adversarial Images
- How Safe Will I Be Given What I Saw? Calibrated Prediction of Safety Chances for Image-Controlled Autonomy
- Marvel: A Data-centric Compiler for DNN Operators on Spatial Accelerators
- Defending Against Image Corruptions Through Adversarial Augmentations
- Agile Autonomous Driving using End-to-End Deep Imitation Learning
- Bayesian Prediction of Future Street Scenes through Importance Sampling based Optimization
- Shared Multi-Task Imitation Learning for Indoor Self-Navigation
- Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving
- Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction
- Reliable and Real-Time Highway Trajectory Planning via Hybrid Learning-Optimization Frameworks
- Cycle-of-Learning for Autonomous Systems from Human Interaction
- Zero-shot Deep Reinforcement Learning Driving Policy Transfer for Autonomous Vehicles based on Robust Control
- A Review of Visual Odometry Methods and Its Applications for Autonomous Driving
- No Need for Interactions: Robust Model-Based Imitation Learning using Neural ODE
- Towards Compact and Robust Deep Neural Networks
- Simple Physical Adversarial Examples against End-to-End Autonomous Driving Models
- An Auto-tuning Framework for Autonomous Vehicles
- Uncertainty Quantification for Large-Scale Deep Networks via Post-StoNet Modeling
- Semantic-Aware Gaussian Process Calibration with Structured Layerwise Kernels for Deep Neural Networks
- FastDriveVLA: Efficient End-to-End Driving via Plug-and-Play Reconstruction-based Token Pruning
- DDD20 End-to-End Event Camera Driving Dataset: Fusing Frames and Events with Deep Learning for Improved Steering Prediction
- StarNet: Targeted Computation for Object Detection in Point Clouds
- A Novel Regression Loss for Non-Parametric Uncertainty Optimization
- Communication-Efficient Distributed Training for Collaborative Flat Optima Recovery in Deep Learning
- DeepObfuscation: Securing the Structure of Convolutional Neural Networks via Knowledge Distillation
- A hybrid controller for safe and efficient collision avoidance control
- Causal Confusion in Imitation Learning
- Recurrent Predictive State Policy Networks
- Are all outliers alike? On Understanding the Diversity of Outliers for Detecting OODs
- Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
- Output Reachable Set Estimation and Verification for Multi-Layer Neural Networks
- A Concept for Efficient Scalability of Automated Driving Allowing for Technical, Legal, Cultural, and Ethical Differences
- AutoPilot: Automating SoC Design Space Exploration for SWaP Constrained Autonomous UAVs
- Live Trojan Attacks on Deep Neural Networks
- Faster CNNs with Direct Sparse Convolutions and Guided Pruning
- Aggressive Deep Driving: Model Predictive Control with a CNN Cost Model
- Learning Nonlinear Causal Reductions to Explain Reinforcement Learning Policies
- TrojDRL: Trojan Attacks on Deep Reinforcement Learning Agents
- Episodic Self-Imitation Learning with Hindsight
- Defining Digital Quadruplets in the Cyber-Physical-Social Space for Parallel Driving
- Intermediate Deep Feature Compression: the Next Battlefield of Intelligent Sensing
- Intention-Net: Integrating Planning and Deep Learning for Goal-Directed Autonomous Navigation
- Detecting Adversarial Samples Using Density Ratio Estimates
- Feature Boosting, Suppression, and Diversification for Fine-Grained Visual Classification
- N 2 C : Neural Network Controller Design Using Behavioral Cloning
- Autonomous Six-Degree-of-Freedom Spacecraft Docking Maneuvers via Reinforcement Learning
- How much real data do we actually need: Analyzing object detection performance using synthetic and real data
- Recurrent Neural Network-based Model for Accelerated Trajectory Analysis in AIMD Simulations
- Behavioral Research and Practical Models of Drivers' Attention
- Adversarial Learning with Margin-based Triplet Embedding Regularization
- Optimizing the Whole-life Cost in End-to-end CNN Acceleration
- ML + FV = \heartsuit? A Survey on the Application of Machine Learning to Formal Verification
- Fighting Failures with FIRE: Failure Identification to Reduce Expert Burden in Intervention-Based Learning
- Gradient-free Policy Architecture Search and Adaptation
- Shaping Rewards for Reinforcement Learning with Imperfect Demonstrations using Generative Models
- Melodic Phrase Segmentation By Deep Neural Networks
- Triple Generative Adversarial Networks
- HySafe-AI: Hybrid Safety Architectural Analysis Framework for AI Systems: A Case Study
- End-to-end driving simulation via angle branched network
- Learning predictive representations in autonomous driving to improve deep reinforcement learning
- Deep Learning Based Motion Planning For Autonomous Vehicle Using Spatiotemporal LSTM Network
- A Survey on Human-aware Robot Navigation
- Tackling Climate Change with Machine Learning
- Neural Co-state Regulator: A Data-Driven Paradigm for Real-time Optimal Control with Input Constraints
- Deep Bilinear Koopman Model for Real-Time Vehicle Control in Frenet Frame
- Adversarial Neural Pruning with Latent Vulnerability Suppression
- End-to-End Deep Learning for Steering Autonomous Vehicles Considering Temporal Dependencies
- InfoGAIL: Interpretable Imitation Learning from Visual Demonstrations
- Self-driving scale car trained by Deep reinforcement learning
- Weakly Supervised Learning Creates a Fusion of Modeling Cultures
- Adaptive MPC for Autonomous Lane Keeping
- Learning Interpretable End-to-End Vision-Based Motion Planning for Autonomous Driving with Optical Flow Distillation
- First-Person Perceptual Guidance Behavior Decomposition using Active Constraint Classification
- Universal adversarial examples in speech command classification
- Adversarial Imitation Learning from Incomplete Demonstrations
- Learning Calibratable Policies using Programmatic Style-Consistency
- End-to-end Interpretable Neural Motion Planner
- DeepRacer: Educational Autonomous Racing Platform for Experimentation with Sim2Real Reinforcement Learning
- Scene-Graph Augmented Data-Driven Risk Assessment of Autonomous Vehicle Decisions
- Hierarchical Policy for Non-prehensile Multi-object Rearrangement with Deep Reinforcement Learning and Monte Carlo Tree Search
- Cooperative Highway Work Zone Merge Control based on Reinforcement Learning in A Connected and Automated Environment
- SAFARI: Safe and Active Robot Imitation Learning with Imagination
- Modular Vehicle Control for Transferring Semantic Information Between Weather Conditions Using GANs
- AI and the Net-Zero Journey: Energy Demand, Emissions, and the Potential for Transition
- MP-RBFN: Learning-based Vehicle Motion Primitives using Radial Basis Function Networks
- Enhanced Transfer Learning for Autonomous Driving with Systematic Accident Simulation
- Counterfactual Visual Explanation via Causally-Guided Adversarial Steering
- Behavioral Exploration: Learning to Explore via In-Context Adaptation
- Exploring the Role of Machine Learning in Scientific Workflows:\n Opportunities and Challenges
- Mirror Learning
- RTGPU: Real-Time Computing with Graphics Processing Units
- Handover Control in Wireless Systems via Asynchronous Multi-User Deep Reinforcement Learning
- Modeling Preemptive Behaviors for Uncommon Hazardous Situations From Demonstrations
- AReN: Assured ReLU NN Architecture for Model Predictive Control of LTI Systems
- One Network to Solve Them All: A Sequential Multi-Task Joint Learning Network Framework for MR Imaging Pipeline
- Learning Interaction-aware Guidance Policies for Motion Planning in Dense Traffic Scenarios
- Counterexample-Guided Learning of Monotonic Neural Networks
- Recall Traces: Backtracking Models for Efficient Reinforcement Learning
- Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
- Disentangling Label Distribution for Long-tailed Visual Recognition
- Adversarial Examples, Uncertainty, and Transfer Testing Robustness in Gaussian Process Hybrid Deep Networks
- Sim2Real Diffusion: Leveraging Foundation Vision Language Models for Adaptive Automated Driving
- Towards End-to-End Deep Learning for Autonomous Racing: On Data Collection and a Unified Architecture for Steering and Throttle Prediction
- QPART: Adaptive Model Quantization and Dynamic Workload Balancing for Accuracy-aware Edge Inference
- Brain Inspired Cognitive Model with Attention for Self-Driving Cars
- A Novel Unsupervised Post-Processing Calibration Method for DNNS with Robustness to Domain Shift
- Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
- Deep Structured Reactive Planning
- A Tour of Reinforcement Learning: The View from Continuous Control
- Learning to drive via Apprenticeship Learning and Deep Reinforcement Learning
- Experimental investigation of pose informed reinforcement learning for skid-steered visual navigation
- Efficient Certified Reasoning for Binarized Neural Networks
- Deep Reinforcement Learning for Human-Like Driving Policies in Collision Avoidance Tasks of Self-Driving Cars
- Neuro-Symbolic Execution: The Feasibility of an Inductive Approach to Symbolic Execution
- Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation
- Inverse reinforcement learning for video games
- Attended Temperature Scaling: A Practical Approach for Calibrating Deep Neural Networks
- One Sample is Enough to Make Conformal Prediction Robust
- Verification of Binarized Neural Networks via Inter-Neuron Factoring
- AI based Algorithms of Path Planning, Navigation and Control for Mobile Ground Robots and UAVs
- Goal-conditioned Hierarchical Reinforcement Learning for Sample-efficient and Safe Autonomous Driving at Intersections
- Learning Task-specific Representation for Novel Words in Sequence Labeling
- Asynchronous "Events" are Better For Motion Estimation
- Convolutional nets for reconstructing neural circuits from brain images acquired by serial section electron microscopy
- Steering Your Diffusion Policy with Latent Space Reinforcement Learning
- Rethinking Semantic Segmentation Evaluation for Explainability and Model Selection
- A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation
- Targeted Nonlinear Adversarial Perturbations in Images and Videos
- Learning Navigation by Visual Localization and Trajectory Prediction
- Pruning and Slicing Neural Networks using Formal Verification
- Feedback from Pixels: Output Regulation via Learning-Based Scene View Synthesis
- DeepRT: A Soft Real Time Scheduler for Computer Vision Applications on the Edge
- Learning to Reach Goals via Iterated Supervised Learning
- Reachable Set Estimation for Neural Network Control Systems: A Simulation-Guided Approach
- Compression Aware Certified Training
- MultiNet: Multi-Modal Multi-Task Learning for Autonomous Driving
- DSDNet: Deep Structured self-Driving Network
- Single Shot MC Dropout Approximation
- Certifying Confidence via Randomized Smoothing
- Behavior Planning at Urban Intersections through Hierarchical Reinforcement Learning
- Hierarchical Feature-level Reverse Propagation for Post-Training Neural Networks
- Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'
- Conditional Driving from Natural Language Instructions
- Spatially-Enhanced Recurrent Memory for Long-Range Mapless Navigation via End-to-End Reinforcement Learning
- DriveAction: A Benchmark for Exploring Human-like Driving Decisions in VLA Models
- Developing a Bubble Chamber Particle Discriminator Using Semi-Supervised Learning
- Policy Learning Using Weak Supervision
- Astra: Toward General-Purpose Mobile Robots via Hierarchical Multimodal Learning
- Incorporating Unlabeled Data into Distributionally Robust Learning
- VisualBackProp for learning using privileged information with CNNs
- Optimal Cooperative Inference
- End-to-end Learning of Driving Models from Large-scale Video Datasets
- Lane Change Decision-making through Deep Reinforcement Learning with Rule-based Constraints
- Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments
- Affordance Learning In Direct Perception for Autonomous Driving
- Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems
- Natural Language Generation with Neural Variational Models
- Accurate Trajectory Prediction for Autonomous Vehicles
- Relational Mimic for Visual Adversarial Imitation Learning
- Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation
- Finite Versus Infinite Neural Networks: an Empirical Study
- Dynamic Defense Approach for Adversarial Robustness in Deep Neural Networks via Stochastic Ensemble Smoothed Model
- Quantitative Projection Coverage for Testing ML-enabled Autonomous Systems
- Federated Imitation Learning: A Privacy Considered Imitation Learning Framework for Cloud Robotic Systems with Heterogeneous Sensor Data
- TASO: Time and Space Optimization for Memory-Constrained DNN Inference
- Requirements for Developing Robust Neural Networks
- Who Make Drivers Stop? Towards Driver-centric Risk Assessment: Risk Object Identification via Causal Inference
- Confidence-Guided Human-AI Collaboration: Reinforcement Learning with Distributional Proxy Value Propagation for Autonomous Driving
- Learning robust driving policies without online exploration
- Decision-Making Technology for Autonomous Vehicles Learning-Based Methods, Applications and Future Outlook
- Reachability Analysis and Safety Verification for Neural Network Control Systems
- SENMAP: Multi-objective data-flow mapping and synthesis for hybrid scalable neuromorphic systems
- An Introduction to Deep Reinforcement Learning
- Safe learning-based optimal motion planning for automated driving
- Vision-Dialog Navigation by Exploring Cross-modal Memory
- Single-step Options for Adversary Driving
- MeLIME: Meaningful Local Explanation for Machine Learning Models
- RealDrive: Retrieval-Augmented Driving with Diffusion Models
- Adversarial Perturbations Against Deep Neural Networks for Malware Classification
- Paracosm: A Language and Tool for Testing Autonomous Driving Systems
- Fast Neural Network Verification via Shadow Prices
- Driving among Flatmobiles: Bird-Eye-View occupancy grids from a monocular camera for holistic trajectory planning
- From WiscKey to Bourbon: A Learned Index for Log-Structured Merge Trees
- A Survey of Behavior Learning Applications in Robotics -- State of the Art and Perspectives
- CogAD: Cognitive-Hierarchy Guided End-to-End Autonomous Driving
- Robust Deep Sensing Through Transfer Learning in Cognitive Radio
- GaussianFusion: Gaussian-Based Multi-Sensor Fusion for End-to-End Autonomous Driving
- Deep Object-Centric Policies for Autonomous Driving
- Learning a Unified Policy for Position and Force Control in Legged Loco-Manipulation
- Rethinking Recurrent Neural Networks and Other Improvements for Image Classification
- SAM: Squeeze-and-Mimic Networks for Conditional Visual Driving Policy Learning
- An Analysis of ISO 26262: Using Machine Learning Safely in Automotive Software
- Defense-guided Transferable Adversarial Attacks
- Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control
- Importance Resampling for Off-policy Prediction
- On Machine Learning and Structure for Mobile Robots
- CIRL: Controllable Imitative Reinforcement Learning for Vision-based Self-driving
- Evolutionary Selective Imitation: Interpretable Agents by Imitation Learning Without a Demonstrator
- Echo Planning for Autonomous Driving: From Current Observations to Future Trajectories and Back
- Better AI through Logical Scaffolding
- An online evolving framework for advancing reinforcement-learning based automated vehicle control
- F1/10: An Open-Source Autonomous Cyber-Physical Platform
- Reach-SDP: Reachability Analysis of Closed-Loop Systems with Neural Network Controllers via Semidefinite Programming
- Graph Neural Lasso for Dynamic Network Regression
- Playing Minecraft with Behavioural Cloning
- Attentional Bottleneck: Towards an Interpretable Deep Driving Network
- Accelerating Targeted Hard-Label Adversarial Attacks in Low-Query Black-Box Settings
- Improving Interpretability of Deep Neural Networks with Semantic Information
- Challenger: Affordable Adversarial Driving Video Generation
- HCRMP: A LLM-Hinted Contextual Reinforcement Learning Framework for Autonomous Driving
- Adaptive Cumulative Mass Calibration with Conformal Prediction
- iPad: Iterative Proposal-centric End-to-End Autonomous Driving
- Long-term Prediction of Vehicle Behavior using Short-term Uncertainty-aware Trajectories and High-definition Maps
- Last Layer Empirical Bayes
- Teaching UAVs to Race: End-to-End Regression of Agile Controls in Simulation
- Automated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learning
- RoadText-1K: Text Detection & Recognition Dataset for Driving Videos
- Composable Action-Conditioned Predictors: Flexible Off-Policy Learning for Robot Navigation
- TBD: Benchmarking and Analyzing Deep Neural Network Training
- Deformation Robust Roto-Scale-Translation Equivariant CNNs
- PRISM: Privileged Probabilistic Latent Supervision for End-to-End Autonomous Driving Motion Planning
- Natural Attribute-based Shift Detection
- Are vision language models robust to uncertain inputs?
- Responsible AI: Gender bias assessment in emotion recognition
- Multi-Vehicle Interaction Scenarios Generation with Interpretable Traffic Primitives and Gaussian Process Regression
- Accelerating Visual-Policy Learning through Parallel Differentiable Simulation
- Automated vehicle's behavior decision making using deep reinforcement learning and high-fidelity simulation environment
- Learning from Maps: Visual Common Sense for Autonomous Driving
- The MineRL 2019 Competition on Sample Efficient Reinforcement Learning using Human Priors
- Sequential Dynamic Decision Making with Deep Neural Nets on a Test-Time Budget
- Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving
- Synthesis of safety certificates for discrete-time uncertain systems via convex optimization
- Meta learning Framework for Automated Driving
- Debona: Decoupled Boundary Network Analysis for Tighter Bounds and Faster Adversarial Robustness Proofs
- MODNet: Moving Object Detection Network with Motion and Appearance for Autonomous Driving
- End-to-End Vision-Based Adaptive Cruise Control (ACC) Using Deep Reinforcement Learning
- How hard is it to cross the room? -- Training (Recurrent) Neural Networks to steer a UAV
- Aggressive Perception-Aware Navigation using Deep Optical Flow Dynamics and PixelMPC
- Sigma-Delta Neural Network Conversion on Loihi 2
- Learning Negotiating Behavior Between Cars in Intersections using Deep Q-Learning
- Learning Resilient Behaviors for Navigation Under Uncertainty
- Optimized Piecewise Affine Abstractions of Neural Networks with Learnable Activation Functions
- DriveNetBench: An Affordable and Configurable Single-Camera Benchmarking System for Autonomous Driving Networks
- Certified Control: An Architecture for Verifiable Safety of Autonomous Vehicles
- Imitation Learning via Off-Policy Distribution Matching
- The NVIDIA PilotNet Experiments
- Mitigating backdoor attacks in LSTM-based Text Classification Systems by Backdoor Keyword Identification
- Unsupervised Temperature Scaling: An Unsupervised Post-Processing Calibration Method of Deep Networks
- Scanner: Efficient Video Analysis at Scale
- Assured Neural Network Architectures for Control and Identification of Nonlinear Systems
- Polarity Loss for Zero-shot Object Detection
- Safety Verification of Deep Neural Networks
- Verification for Machine Learning, Autonomy, and Neural Networks Survey
- Demystifying Parallel and Distributed Deep Learning
- There is Limited Correlation between Coverage and Robustness for Deep Neural Networks
- Decision-Making under On-Ramp merge Scenarios by Distributional Soft Actor-Critic Algorithm
- PODNet: A Neural Network for Discovery of Plannable Options
- Accelerated Convolutions for Efficient Multi-Scale Time to Contact Computation in Julia
- Periphery-Fovea Multi-Resolution Driving Model guided by Human Attention
- Design Space of Behaviour Planning for Autonomous Driving
- A review of green artificial intelligence: Towards a more sustainable future
- Unleashing the Potential of Diffusion Models for End-to-End Autonomous Driving
- Learning a Safety Verifiable Adaptive Cruise Controller from Human Driving Data
- DeepRoad: GAN-based Metamorphic Autonomous Driving System Testing
- Maximum Likelihood Reinforcement Learning
- Learning to Drive from a World Model
- Imitation Learning for Autonomous Driving: Insights from Real-World Testing
- Transferring Autonomous Driving Knowledge on Simulated and Real\n Intersections
- Explorations and Lessons Learned in Building an Autonomous Formula SAE Car from Simulations
- Toward Fully Autonomous Driving: AI, Challenges, Opportunities, and Needs
- Autonomous Cars: Vision based Steering Wheel Angle Estimation
- Timing Attacks on Machine Learning: State of the Art
- Data Analytics Service Composition and Deployment on Edge Devices
- Understanding Multi-Modal Perception Using Behavioral Cloning for Peg-In-a-Hole Insertion Tasks
- Analyzing Representations inside Convolutional Neural Networks
- Interpretable Fuzzy Inference for UAV Target Tracking Using Bounding-Box Geometry
- Investigating Deep Learning Methods for Obtaining Photometric Redshift Estimations from Images
- CARLA Real Traffic Scenarios -- novel training ground and benchmark for autonomous driving
- A Behavioral Approach to Visual Navigation with Graph Localization Networks
- Large scale visual place recognition with sub-linear storage growth
- Verifying Aircraft Collision Avoidance Neural Networks Through Linear Approximations of Safe Regions
- Comparing Apples and Oranges: Off-Road Pedestrian Detection on the NREC Agricultural Person-Detection Dataset
- Practical Convex Formulation of Robust One-hidden-layer Neural Network Training
- Per-Pixel Feedback for improving Semantic Segmentation
- A survey of deep learning techniques for autonomous driving
- Merging in Congested Freeway Traffic Using Multipolicy Decision Making and Passive Actor-Critic Learning
- Using Machine Learning Safely in Automotive Software: An Assessment and\n Adaption of Software Process Requirements in ISO 26262
- A Systematic Comparison of Deep Learning Architectures in an Autonomous Vehicle
- Data Scaling Laws for End-to-End Autonomous Driving
- An Online Evolving Framework for Modeling the Safe Autonomous Vehicle Control System via Online Recognition of Latent Risks
- Reachable Set Computation and Safety Verification for Neural Networks with ReLU Activations
- Fortified Networks: Improving the Robustness of Deep Networks by Modeling the Manifold of Hidden Representations
- PLOP: Probabilistic poLynomial Objects trajectory Planning for autonomous driving
- Exposing the Copycat Problem of Imitation-based Planner: A Novel Closed-Loop Simulator, Causal Benchmark and Joint IL-RL Baseline
- Learning Through Retrospection: Improving Trajectory Prediction for Automated Driving with Error Feedback
- The Importance of Balanced Data Sets: Analyzing a Vehicle Trajectory Prediction Model based on Neural Networks and Distributed Representations
- On Evaluation of Adversarial Perturbations for Sequence-to-Sequence\n Models
- Benchmark and application of unsupervised classification approaches for univariate data
- Support is All You Need for Certified VAE Training
- Data-based stabilization of unknown bilinear systems with guaranteed basin of attraction
- Towards Calibration Enhanced Network by Inverse Adversarial Attack
Discussions
Related