Human-centric Reward Optimization for Reinforcement Learning-based Automated Driving using Large Language Models

2024/05/07 by Jingyue Zhang, Zhou, Ziqi, Jingyuan Zhang +8 · 3 citations
Computer Science · Engineering · #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Autonomous Vehicle Technology and Safety #FOS: Computer and information sciences #Reinforcement Learning in Robotics

paper · pdf · doi:10.48550/arxiv.2405.04135

openalex publication_date 2024/05/07 · openalex created_date 2024/05/10 · openalex updated_date 2026/07/28

Abstract

One of the key challenges in current Reinforcement Learning (RL)-based Automated Driving (AD) agents is achieving flexible, precise, and human-like behavior cost-effectively. This paper introduces an innovative approach that uses large language models (LLMs) to intuitively and effectively optimize RL reward functions in a human-centric way. We developed a framework where instructions and dynamic environment descriptions are input into the LLM. The LLM then utilizes this information to assist in generating rewards, thereby steering the behavior of RL agents towards patterns that more closely resemble human driving. The experimental results demonstrate that this approach not only makes RL agents more anthropomorphic but also achieves better performance. Additionally, various strategies for reward-proxy and reward-shaping are investigated, revealing the significant impact of prompt design on shaping an AD vehicle's behavior. These findings offer a promising direction for the development of more advanced, human-like automated driving systems. Our experimental data and source code can be found here

Cited by

• Are LLMs The Way Forward? A Case Study on LLM-Guided Reinforcement Learning for Decentralized Autonomous Driving
• ASTGI: Adaptive Spatio-Temporal Graph Interactions for Irregular Multivariate Time Series Forecasting
• Multi-Modal Machine Learning Framework for Predicting Early Recurrence of Brain Tumors Using MRI and Clinical Biomarkers

Related

• Generating and Evolving Reward Functions for Highway Driving with Large Language Models
• LORD: Large Models based Opposite Reward Design for Autonomous Driving
• REvolve: Reward Evolution with Large Language Models using Human Feedback
• Large Language Models for Human-like Autonomous Driving: A Survey
• CLIP-RLDrive: Human-Aligned Autonomous Driving via CLIP-Based Reward Shaping in Reinforcement Learning