From Fields to Splats: A Cross-Domain Survey of Real-Time Neural Scene Representations
2025/09/28 by Ahmad, Javed, Gao, Penggang, Delehelle, Donatien +5
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences
paper · doi:10.48550/arxiv.2509.23555
Abstract
Neural scene representations such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have transformed how 3D environments are modeled, rendered, and interpreted. NeRF introduced view-consistent photorealism via volumetric rendering; 3DGS has rapidly emerged as an explicit, efficient alternative that supports high-quality rendering, faster optimization, and integration into hybrid pipelines for enhanced photorealism and task-driven scene understanding. This survey examines how 3DGS is being adopted across SLAM, telepresence and teleoperation, robotic manipulation, and 3D content generation. Despite their differences, these domains share common goals: photorealistic rendering, meaningful 3D structure, and accurate downstream tasks. We organize the review around unified research questions that explain why 3DGS is increasingly displacing NeRF-based approaches: What technical advantages drive its adoption? How does it adapt to different input modalities and domain-specific constraints? What limitations remain? By systematically comparing domain-specific pipelines, we show that 3DGS balances photorealism, geometric fidelity, and computational efficiency. The survey offers a roadmap for leveraging neural rendering not only for image synthesis but also for perception, interaction, and content creation across real and virtual environments.
Citations
- Surgical Neural Radiance Fields from One Image
- VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold
- NeRF-Based Transparent Object Grasping Enhanced by Shape Priors
- Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction
- ExScene: Free-View 3D Scene Reconstruction with Gaussian Splatting from a Single Image
- Aether: Geometric-Aware Unified World Modeling
- Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors
- VGGT: Visual Geometry Grounded Transformer
- Persistent Object Gaussian Splat (POGS) for Tracking Human and Robot Manipulation of Irregularly Shaped Objects
- FLARE: Feed-forward Geometry, Appearance and Camera Estimation from Uncalibrated Sparse Views
- DenseSplat: Densifying Gaussian Splatting SLAM with Neural Radiance Prior
- Re3Sim: Generating High-Fidelity Simulation Data via 3D-Photorealistic Real-to-Sim for Robotic Manipulation
- TranSplat: Surface Embedding-guided 3D Gaussian Splatting for Transparent Object Manipulation
- GARAD-SLAM: 3D GAussian splatting for Real-time Anti Dynamic SLAM
- Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass
- Neural Radiance Fields for the Real World: A Survey
- Continuous 3D Perception Model with Persistent State
- Safe Dynamic Motion Generation in Configuration Space Using Differentiable Distance Fields
- MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors
- Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization
- Advancing Extended Reality with 3D Gaussian Splatting: Innovations and Prospects
- MAGiC-SLAM: Multi-Agent Gaussian Globally Consistent SLAM
- PreF3R: Pose-Free Feed-Forward 3D Gaussian Splatting from Variable-length Image Sequence
- Baking Gaussian Splatting into Diffusion Denoiser for Fast and Scalable Single-stage Image-to-3D Generation and Reconstruction
- Neural Fields in Robotics: A Survey
- GARField: Addressing the visual Sim-to-Real gap in garment manipulation with mesh-attached radiance fields
- MonST3R: A Simple Approach for Estimating Geometry in the Presence of Motion
- Robust Incremental Structure-from-Motion with Hybrid Features
- MASt3R-SfM: a Fully-Integrated Solution for Unconstrained Structure-from-Motion
- SwinGS: Sliding Window Gaussian Splatting for Volumetric Video Streaming with Arbitrary Length
- Robust Dual Gaussian Splatting for Immersive Human-centric Volumetric Videos
- Safe and Efficient Path Planning under Uncertainty via Deep Collision Probability Fields
- GraspSplats: Efficient Manipulation with 3D Feature Splatting
- 3D Reconstruction with Spatial Memory
- Splatt3R: Zero-shot Gaussian Splatting from Uncalibrated Image Pairs
- LoopSplat: Loop Closure by Registering 3D Gaussian Splats
- Reality Fusion: Robust Real-time Immersive Mobile Robot Teleoperation with Volumetric Visual Data Fusion
- IG-SLAM: Instant Gaussian SLAM
- SceneTeller: Language-to-3D Scene Generation
- Global Structure-from-Motion Revisited
- Radiance Fields for Robotic Teleoperation
- 3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities
- 3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities
- HoloDreamer: Holistic 3D Panoramic World Generation from Text Descriptions
- Connecting Consistency Distillation to Score Distillation for Text-to-3D Generation
- Grounding Image Matching in 3D with MASt3R
- WonderWorld: Interactive 3D Scene Generation from a Single Image
- 4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models
- Flash3D: Feed-Forward Generalisable 3D Scene Reconstruction from a Single Image
- EG4D: Explicit Generation of 4D Object without Score Distillation
- Splat-SLAM: Globally Optimized RGB-only SLAM with 3D Gaussians
- Tele-Aloha: A Low-budget and High-authenticity Telepresence System Using Sparse RGB Cameras
- ART3D: 3D Gaussian Splatting for Text-Guided Artistic Scenes Generation
- Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splatting
- DreamScene4D: Dynamic Multi-Object Scene Generation from Monocular Videos
- RealmDreamer: Text-Driven 3D Scene Generation with Inpainting and Depth Diffusion
- DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splatting
- DreamScene: 3D Gaussian-based Text-to-3D Scene Generation via Formation Pattern Sampling
- GlORIE-SLAM: Globally Optimized RGB-only Implicit Encoding Point Cloud SLAM
- Comp4D: LLM-Guided Compositional 4D Scene Generation
- GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation
- RGBD GS-ICP SLAM
- BrightDreamer: Generic 3D Gaussian Generative Framework for Fast Text-to-3D Synthesis
- Controllable Text-to-3D Generation via Surface-Aligned Gaussian Splatting
- GaussianGrasper: 3D Language Gaussian Splatting for Open-vocabulary Robotic Grasping
- ManiGaussian: Dynamic Gaussian Splatting for Multi-task Robotic Manipulation
- SemGauss-SLAM: Dense Semantic Gaussian Splatting SLAM
- Closing the Visual Sim-to-Real Gap with Object-Composable NeRFs
- Splat-Nav: Safe Real-Time Robot Navigation in Gaussian Splatting Maps
- How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey
- LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation
- VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality
- Efficient4D: Fast Dynamic 3D Object Generation from a Single-view Video
- DreamGaussian4D: Generative 4D Gaussian Splatting
- DUSt3R: Geometric 3D Vision Made Easy
- Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models
- Text2Immersion: Generative Immersive Scene with 3D Gaussians
- Gaussian Splatting SLAM
- Visual Geometry Grounded Deep Structure From Motion
- SplaTAM: Splat, Track & Map 3D Gaussians for Dense RGB-D SLAM
- CG3D: Compositional Generation for Text-to-3D via Gaussian Splatting
- Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D Cameras
- GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting
- Consistent4D: Consistent 360° Dynamic Object Generation from Monocular Video
- Stochastic Implicit Neural Signed Distance Functions for Safe Motion Planning under Sensing Uncertainty
- GO-SLAM: Global Optimization for Consistent 3D Instant Reconstruction
- 3D Gaussian Splatting for Real-Time Radiance Field Rendering
- Representing Robot Geometry as Distance Fields: Applications to Whole-body Manipulation
- Collision-free Motion Generation Based on Stochastic Optimization and Composite Signed Distance Field Networks of Articulated Robot
- Text2NeRF: Text-Driven 3D Scene Generation with Neural Radiance Fields
- Micrograph segmentations for DDEVD
- Segment Anything
- ClusterGNN: Cluster-based Coarse-to-Fine Graph Neural Network for Efficient Feature Matching
- Regularized Deep Signed Distance Fields for Reactive Motion Generation
- NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
- Zero-Shot Text-Guided Object Generation with Dream Fields
- Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objects
- Learning to Match Features with Seeded Graph Matching Network
- Perceiver IO: A General Architecture for Structured Inputs & Outputs
- Deep Two-View Structure-from-Motion Revisited
- Vision Transformers for Dense Prediction
- iMAP: Implicit Mapping and Positioning in Real-Time
- Learning Transferable Visual Models From Natural Language Supervision
- INeRF: Inverting Neural Radiance Fields for Pose Estimation
- DISK: Learning local features with policy gradient
- SuperGlue: Learning Feature Matching with Graph Neural Networks
- DeepSFM: Structure From Motion Via Deep Bundle Adjustment
- kPAM-SC: Generalizable Manipulation Planning using KeyPoint Affordance\n and Shape Completion
- D2-Net: A Trainable CNN for Joint Detection and Description of Local\n Features
- DeepV2D: Video to Depth with Differentiable Structure from Motion
- BA-Net: Dense Bundle Adjustment Network
- SuperPoint: Self-Supervised Interest Point Detection and Description
- PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
- LIFT: Learned Invariant Feature Transform
- Dynamics-Aware Gaussian Splatting Streaming Towards Fast On-the-Fly 4D Reconstruction
Related