2024/12/13 by Wenzhao Zheng, Zheng, Wenzhao, Junjie Wu +17 · 12 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Autonomous Vehicle Technology and Safety #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Robotic Path Planning Algorithms #Robotics (cs.RO)
paper · pdf · doi:10.48550/arxiv.2412.10371
openalex publication_date 2024/12/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Vision-based autonomous driving shows great potential due to its satisfactory\nperformance and low costs. Most existing methods adopt dense representations\n(e.g., bird's eye view) or sparse representations (e.g., instance boxes) for\ndecision-making, which suffer from the trade-off between comprehensiveness and\nefficiency. This paper explores a Gaussian-centric end-to-end autonomous\ndriving (GaussianAD) framework and exploits 3D semantic Gaussians to\nextensively yet sparsely describe the scene. We initialize the scene with\nuniform 3D Gaussians and use surrounding-view images to progressively refine\nthem to obtain the 3D Gaussian scene representation. We then use sparse\nconvolutions to efficiently perform 3D perception (e.g., 3D detection, semantic\nmap construction). We predict 3D flows for the Gaussians with dynamic semantics\nand plan the ego trajectory accordingly with an objective of future scene\nforecasting. Our GaussianAD can be trained in an end-to-end manner with\noptional perception labels when available. Extensive experiments on the widely\nused nuScenes dataset verify the effectiveness of our end-to-end GaussianAD on\nvarious tasks including motion planning, 3D occupancy prediction, and 4D\noccupancy forecasting. Code: https://github.com/wzzheng/GaussianAD.\n