Trajectory Planning for Autonomous Vehicles Using Hierarchical\n Reinforcement Learning

2020/11/09 by Kaleb Ben Naveed, Naveed, Kaleb Ben, Zhiqian Qiao +3 · 9 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Artificial intelligence #Autonomous Vehicle Technology and Safety #Computer science #Control (management) #Control engineering #Control theory (sociology) #Controller (irrigation) #Convergence (economics) #Engineering #FOS: Computer and information sciences #Mobile robot #Motion planning #Obstacle avoidance #PID controller #Planner #Reinforcement Learning in Robotics #Reinforcement learning #Robot #Robotic Path Planning Algorithms #Robotics (cs.RO) #Trajectory #cs.AI #cs.RO

paper · pdf · doi:10.48550/arxiv.2011.04752

published in arXiv (Cornell University) (Cornell University) · 7 pages, 5 figures

arxiv created 2020/11/09 · openalex publication_date 2020/11/09 · arxiv updated 2020/11/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/06

Abstract

Planning safe trajectories under uncertain and dynamic conditions makes the\nautonomous driving problem significantly complex. Current sampling-based\nmethods such as Rapidly Exploring Random Trees (RRTs) are not ideal for this\nproblem because of the high computational cost. Supervised learning methods\nsuch as Imitation Learning lack generalization and safety guarantees. To\naddress these problems and in order to ensure a robust framework, we propose a\nHierarchical Reinforcement Learning (HRL) structure combined with a\nProportional-Integral-Derivative (PID) controller for trajectory planning. HRL\nhelps divide the task of autonomous vehicle driving into sub-goals and supports\nthe network to learn policies for both high-level options and low-level\ntrajectory planner choices. The introduction of sub-goals decreases convergence\ntime and enables the policies learned to be reused for other scenarios. In\naddition, the proposed planner is made robust by guaranteeing smooth\ntrajectories and by handling the noisy perception system of the ego-car. The\nPID controller is used for tracking the waypoints, which ensures smooth\ntrajectories and reduces jerk. The problem of incomplete observations is\nhandled by using a Long-Short-Term-Memory (LSTM) layer in the network. Results\nfrom the high-fidelity CARLA simulator indicate that the proposed method\nreduces convergence time, generates smoother trajectories, and is able to\nhandle dynamic surroundings and noisy observations.\n

Cited by

• DTCCL: Disengagement-Triggered Contrastive Continual Learning for Autonomous Bus Planners
• A Review of Learning-Based Motion Planning: Toward a Data-Driven Optimal Control Approach
• HAVEN: Hierarchical Adversary-aware Visibility-Enabled Navigation with Cover Utilization using Deep Transformer Q-Networks
• Learning to Drive Safely with Hybrid Options
• CoMoCAVs: Cohesive Decision-Guided Motion Planning for Connected and Autonomous Vehicles with Multi-Policy Reinforcement Learning
• GenAI-based Multi-Agent Reinforcement Learning towards Distributed Agent Intelligence: A Generative-RL Agent Perspective
• Ego-centric Learning of Communicative World Models for Autonomous Driving
• Automatic Curriculum Learning for Driving Scenarios: Towards Robust and Efficient Reinforcement Learning
• Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving

Related

• A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning
• Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning
• Learning hierarchical behavior and motion planning for autonomous driving
• Trajectory Planning for Autonomous Vehicle Using Iterative Reward Prediction in Reinforcement Learning
• Safe Trajectory Planning Using Reinforcement Learning for Self Driving