2024/12/13 by Sicheng Zuo, Zuo, Sicheng, Wenzhao Zheng +7 · 22 citations
Computer Science · #Data Management and Algorithms #Video Surveillance and Tracking Methods #Advanced Clustering Algorithms Research
paper · pdf · doi:10.48550/arxiv.2412.10373
3D occupancy prediction is important for autonomous driving due to its\ncomprehensive perception of the surroundings. To incorporate sequential inputs,\nmost existing methods fuse representations from previous frames to infer the\ncurrent 3D occupancy. However, they fail to consider the continuity of driving\nscenarios and ignore the strong prior provided by the evolution of 3D scenes\n(e.g., only dynamic objects move). In this paper, we propose a\nworld-model-based framework to exploit the scene evolution for perception. We\nreformulate 3D occupancy prediction as a 4D occupancy forecasting problem\nconditioned on the current sensor input. We decompose the scene evolution into\nthree factors: 1) ego motion alignment of static scenes; 2) local movements of\ndynamic objects; and 3) completion of newly-observed scenes. We then employ a\nGaussian world model (GaussianWorld) to explicitly exploit these priors and\ninfer the scene evolution in the 3D Gaussian space considering the current RGB\nobservation. We evaluate the effectiveness of our framework on the widely used\nnuScenes dataset. Our GaussianWorld improves the performance of the\nsingle-frame counterpart by over 2% in mIoU without introducing additional\ncomputations. Code: https://github.com/zuosc19/GaussianWorld.\n