NeRF
2021/12/17 by Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik +3 · 6,079 citations
Computer Science · Engineering · Mathematics · #3D Shape Modeling and Analysis #Advanced Vision and Imaging #Artificial intelligence #Artificial neural network #Computer Graphics and Visualization Techniques #Computer graphics (images) #Computer science #Computer vision #Differentiable function #Image-based modeling and rendering #Mathematics #Optics #Physics #Radiance #Rendering (computer graphics) #View synthesis #Volume rendering
paper · pdf · doi:10.1145/3503250
published in Communications of the ACM 65(1), 99-106 (Association for Computing Machinery)
openalex publication_date 2021/12/17 · openalex created_date 2021/12/31 · openalex updated_date 2026/08/05
Abstract
We present a method that achieves state-of-the-art results for synthesizing novel views of complex scenes by optimizing an underlying continuous volumetric scene function using a sparse set of input views. Our algorithm represents a scene using a fully connected (nonconvolutional) deep network, whose input is a single continuous 5D coordinate (spatial location ( x , y , z ) and viewing direction ( θ, ϕ )) and whose output is the volume density and view-dependent emitted radiance at that spatial location. We synthesize views by querying 5D coordinates along camera rays and use classic volume rendering techniques to project the output colors and densities into an image. Because volume rendering is naturally differentiable, the only input required to optimize our representation is a set of images with known camera poses. We describe how to effectively optimize neural radiance fields to render photorealistic novel views of scenes with complicated geometry and appearance, and demonstrate results that outperform prior work on neural rendering and view synthesis.
Cited by
- NPGA: Neural Parametric Gaussian Avatars
- WonderHuman: Hallucinating Unseen Parts in Dynamic 3D Human Reconstruction
- Single-molecule orientation-localization microscopy: Applications and approaches
- IncepHoloRGB: multi-wavelength network model for full-color 3D computer-generated holography
- 3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities
- Foram3D: A pipeline for 3D synthetic data generation and rendering of foraminifera for image analysis and reconstruction
- InstaMap: instant-NGP for cryo-EM density maps
- Plant3R: Fusing 3D feature learning with Gaussian splatting to enhance wheat plant 3D reconstruction precision
- High-fidelity wheat plant reconstruction using 3D Gaussian splatting and neural radiance fields
- Survey of FOSS 3D/2D graphics software Blender usage in science, academia, and industry
- Estimating Iceberg Melt Rates from Video Using Novel View Synthesis
- Deep learning in plant phenotyping: the first ten years
- New era towards autonomous additive manufacturing: a review of recent trends and future perspectives
- Autonomy 2.0: The Quest for Economies of Scale
- NExpR: Neural Explicit Representation for fast arbitrary-scale medical image super-resolution
- Convolutional neural shading for high-quality 3D reconstruction from multi-view images
- Collaborative feature aggregation for face super-resolution and robust re-identification
- NeVF: Representing CFD simulations as neural flow volume fields for efficient compression, reconstruction, and analysis
- SCAG: A Stratified, Clustered, and Growing-Based Algorithm for Soybean Branch Angle Extraction and Ideal Plant Architecture Evaluation
- HoloHand: enhancing real-time neural hand rendering with self-occlusion-aware appearance fields and depth-aware learning
- DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance
- Li-GS: a fast 3D Gaussian reconstruction method assisted by LiDAR point clouds
- Point cloud upsampling with implicit graph neural networks
- MetaSapiens: Real-Time Neural Rendering with Efficiency-Aware Pruning and Accelerated Foveated Rendering
- Learning agile soccer skills for a bipedal robot with deep reinforcement learning
- Random-Access Neural Compression of Material Textures
- FAST-GS: Frequency Aware Space-Time Gaussian Splatting for Photorealistic Dynamic Novel View Synthesis
- Embodied Object Representation Learning and Recognition. [europepmc]
- Recent Advances in Deep Learning for Protein-Protein Interaction Analysis: A Comprehensive Review. [europepmc]
- Measurement Technologies of Light Field Camera: An Overview. [europepmc]
- A Survey of 6DoF Object Pose Estimation Methods for Different Application Scenarios. [europepmc]
- Single-pixel p-graded-n junction spectrometers. [europepmc]
- 3D cine-magnetic resonance imaging using spatial and temporal implicit neural representation learning (STINR-MR). [europepmc]
- PA-NeRF, a neural radiance field model for 3D photoacoustic tomography reconstruction from limited Bscan data. [europepmc]
- Beyond Discrimination: Generative AI Applications and Ethical Challenges in Forensic Psychiatry. [europepmc]
- A Comprehensive Review of Vision-Based 3D Reconstruction Methods. [europepmc]
- Dynamic CBCT imaging using prior model-free spatiotemporal implicit neural representation (PMF-STINR). [europepmc]
- Visual SLAM for Unmanned Aerial Vehicles: Localization and Perception. [europepmc]
- Deep learning for accelerated and robust MRI reconstruction. [europepmc]
- Evolutionary Implications of Self-Assembling Cybernetic Materials with Collective Problem-Solving Intelligence at Multiple Scales. [europepmc]
- DynaMight: estimating molecular motions with improved reconstruction from cryo-EM images. [europepmc]
- Neural space-time model for dynamic multi-shot imaging. [europepmc]
- Scaling and merging time-resolved pink-beam diffraction with variational inference. [europepmc]
- Single-molecule orientation-localization microscopy: Applications and approaches. [europepmc]
- Real-time CBCT imaging and motion tracking via a single arbitrarily-angled x-ray projection by a joint dynamic reconstruction and motion estimation (DREME) framework. [europepmc]
- Neural Radiance Fields (NeRF) for 3D Reconstruction of Monocular Endoscopic Video in Sinus Surgery. [europepmc]
- Generative artificial intelligence: a historical perspective. [europepmc]
- Plant stem and leaf segmentation and phenotypic parameter extraction using neural radiance fields and lightweight point cloud segmentation networks. [europepmc]
- Shorter SPECT scans using self-supervised coordinate learning to synthesize skipped projection views. [europepmc]
- Single-shot reconstruction of three-dimensional morphology of biological cells in digital holographic microscopy using a physics-driven neural network. [europepmc]
- A Three-Dimensional Phenotype Extraction Method Based on Point Cloud Segmentation for All-Period Cotton Multiple Organs. [europepmc]
- Fourier-based three-dimensional multistage transformer for aberration correction in multicellular specimens. [europepmc]
Related