Inverse reinforcement learning for autonomous navigation via differentiable semantic mapping and planning

2021/01/01 by Tianyu Wang, Vikas Dhiman, Wang, Tianyu +3
Computer Science · Engineering · #Autonomous Vehicle Technology and Safety #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Robotic Path Planning Algorithms #Robotics (cs.RO) #cs.LG #cs.RO

paper · pdf · doi:10.48550/arxiv.2101.00186

16 pages, 12 figures. arXiv admin note: text overlap with arXiv:2006.05043

arxiv created 2021/01/01 · openalex publication_date 2021/01/01 · arxiv updated 2021/01/05 · openalex created_date 2021/01/18 · openalex updated_date 2026/07/28

Abstract

This paper focuses on inverse reinforcement learning for autonomous navigation using distance and semantic category observations. The objective is to infer a cost function that explains demonstrated behavior while relying only on the expert's observations and state-control trajectory. We develop a map encoder, that infers semantic category probabilities from the observation sequence, and a cost encoder, defined as a deep neural network over the semantic features. Since the expert cost is not directly observable, the model parameters can only be optimized by differentiating the error between demonstrated controls and a control policy computed from the cost estimate. We propose a new model of expert behavior that enables error minimization using a closed-form subgradient computed only over a subset of promising states via a motion planning algorithm. Our approach allows generalizing the learned behavior to new environments with new spatial configurations of the semantic categories. We analyze the different components of our model in a minigrid environment. We also demonstrate that our approach learns to follow traffic rules in the autonomous driving CARLA simulator by relying on semantic observations of buildings, sidewalks, and road lanes.

Citations

• Learning Navigation Costs from Demonstration with Semantic Observations
• Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
• A survey of robot learning from demonstration
• Apprenticeship Learning using Inverse Reinforcement Learning and Gradient Methods
• A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models
• Reinforcement Learning with Deep Energy-Based Policies
• Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
• A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Driving

Related

• Learning Navigation Costs from Demonstration with Semantic Observations
• Learning Navigation Costs from Demonstration in Partially Observable Environments
• Learning Risk-Aware Costmaps via Inverse Reinforcement Learning for Off-Road Navigation
• Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Driving
• Approximate Inverse Reinforcement Learning from Vision-based Imitation Learning