2024/11/15 by Sonda Fourati, Wael Jaafar, Fourati, Sonda +3
Engineering · Environmental Science · #Air Quality Monitoring and Forecasting #FOS: Computer and information sciences #FOS: Electrical engineering #Robotics (cs.RO) #Robotics and Automated Systems #Systems and Control (eess.SY) #Traffic Prediction and Management Techniques #electronic engineering #information engineering
paper · pdf · doi:10.48550/arxiv.2411.10603
openalex publication_date 2024/11/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Autonomous driving (AD) technology promises to revolutionize daily transportation by making it safer, more efficient, and more comfortable. Their role in reducing traffic accidents and improving mobility will be vital to the future of intelligent transportation systems. Autonomous driving in harsh environmental conditions presents significant challenges that demand robust and adaptive solutions and require more investigation. In this context, we present in this paper a comprehensive performance analysis of an autonomous driving agent leveraging the capabilities of a Multi-modal Large Language Model (MLLM) using GPT-4o within the LimSim++ framework that offers close loop interaction with the CARLA driving simulator. We call it MLLM-AD-4o. Our study evaluates the agent's decision-making, perception, and control under adverse conditions, including bad weather, poor visibility, and complex traffic scenarios. Our results demonstrate the AD agent's ability to maintain high levels of safety and efficiency, even in challenging environments, underscoring the potential of GPT-4o to enhance autonomous driving systems (ADS) in any environment condition. Moreover, we evaluate the performance of MLLM-AD-4o when different perception entities are used including either front cameras only, front and rear cameras, and when combined with LiDAR. The results of this work provide valuable insights into integrating MLLMs with AD frameworks, paving the way for future advancements in this field.