BEV-VLM: Trajectory Planning via Unified BEV Abstraction

2025/09/27 by Guancheng Chen, Chen, Guancheng, Yang, Sheng +4
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Autonomous Vehicle Technology and Safety #FOS: Computer and information sciences #Robotic Path Planning Algorithms #Robotics (cs.RO) #Traffic Prediction and Management Techniques

paper · pdf · doi:10.48550/arxiv.2509.25249

openalex publication_date 2025/09/27 · openalex created_date 2025/10/19 · openalex updated_date 2026/07/28

Abstract

This paper introduces BEV-VLM, a novel approach for trajectory planning in autonomous driving that leverages Vision-Language Models (VLMs) with Bird's-Eye View (BEV) feature maps as visual input. Unlike conventional trajectory planning approaches that rely solely on raw visual data (e.g., camera images), our method utilizes a highly compressed and informative BEV representation generated by fusing camera and LiDAR data, with subsequent alignment to High-Definition (HD) maps. This unified BEV-HD map format provides a geometrically consistent and semantically rich scene description, which enables VLMs to perform accurate and robust trajectory planning. Experimental results on the nuScenes dataset demonstrate that, compared with state-of-the-art vision-only methods, our approach achieves a 53.1% improvement in planning accuracy and realizes complete collision avoidance in evaluation scenarios. Our work highlights that VLMs can effectively interpret processed visual representations such as BEV features, expanding their applicability beyond raw image inputs for the task of trajectory planning.

Citations

• AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
• OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model
• Qwen2.5-VL Technical Report
• VLM-AD: End-to-End Autonomous Driving through Vision-Language Model Supervision
• EMMA: End-to-End Multimodal Model for Autonomous Driving
• Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving
• VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning
• Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving
• DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model
• VAD: Vectorized Scene Representation for Efficient Autonomous Driving
• Planning-oriented Autonomous Driving
• ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learning
• BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representation
• BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving
• BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers
• Lift, Splat, Shoot: Encoding Images From Arbitrary Camera Rigs by Implicitly Unprojecting to 3D
• nuScenes: A multimodal dataset for autonomous driving
• OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning

Related

• BEVDriver: Leveraging BEV Maps in LLMs for Robust Closed-Loop Driving
• BEVTraj: Map-Free End-to-End Trajectory Prediction in Bird's-Eye View with Deformable Attention and Sparse Goal Proposals
• Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving
• BEV-LLM: Leveraging Multimodal BEV Maps for Scene Captioning in Autonomous Driving
• SimBEV: A Synthetic Multi-Task Multi-Sensor Driving Data Generation Tool and Dataset