vix.ing · top · new · best · stats

4D Gaussian Splatting for Real-Time Dynamic Scene Rendering

2023/10/12 by Guanjun Wu, Taoran Yi, Wu, Guanjun +15 · 352 citations
Computer Science · #Advanced Vision and Imaging #Computer Graphics and Visualization Techniques #Human Pose and Action Recognition

paper · pdf · doi:10.48550/arxiv.2310.08528

Abstract

Representing and rendering dynamic scenes has been an important but challenging task. Especially, to accurately model complex motions, high efficiency is usually hard to guarantee. To achieve real-time dynamic scene rendering while also enjoying high training and storage efficiency, we propose 4D Gaussian Splatting (4D-GS) as a holistic representation for dynamic scenes rather than applying 3D-GS for each individual frame. In 4D-GS, a novel explicit representation containing both 3D Gaussians and 4D neural voxels is proposed. A decomposed neural voxel encoding algorithm inspired by HexPlane is proposed to efficiently build Gaussian features from 4D neural voxels and then a lightweight MLP is applied to predict Gaussian deformations at novel timestamps. Our 4D-GS method achieves real-time rendering under high resolutions, 82 FPS at an 800×800 resolution on an RTX 3090 GPU while maintaining comparable or better quality than previous state-of-the-art methods. More demos and code are available at https://guanjunwu.github.io/4dgs/.

Cited by

Related