ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenes

2023/08/22 by Chandan Yeshwanth, Yueh-Cheng Liu, Yeshwanth, Chandan +5 · 147 citations
Computer Science · Earth and Planetary Sciences · Environmental Science · #3D Surveying and Cultural Heritage #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Remote Sensing and LiDAR Applications

paper · pdf · doi:10.48550/arxiv.2308.11417

openalex publication_date 2023/08/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present ScanNet++, a large-scale dataset that couples together capture of high-quality and commodity-level geometry and color of indoor scenes. Each scene is captured with a high-end laser scanner at sub-millimeter resolution, along with registered 33-megapixel images from a DSLR camera, and RGB-D streams from an iPhone. Scene reconstructions are further annotated with an open vocabulary of semantics, with label-ambiguous scenarios explicitly annotated for comprehensive semantic understanding. ScanNet++ enables a new real-world benchmark for novel view synthesis, both from high-quality RGB capture, and importantly also from commodity-level images, in addition to a new benchmark for 3D semantic scene understanding that comprehensively encapsulates diverse and ambiguous semantic labeling scenarios. Currently, ScanNet++ contains 460 scenes, 280,000 captured DSLR images, and over 3.7M iPhone RGBD frames.

Cited by

• SpatialMosaic: A Multiview VLM Dataset for Partial Visibility
• 3D sans 3D Scans: Scalable Pre-training from Video-Generated Point Clouds
• ShapeR: Robust Conditional 3D Shape Generation from Casual Captures
• Unlocking the Power of Critical Factors for 3D Visual Geometry Estimation
• GeoAnchor: Collaborative Reasoning via Latent Decomposition for 3D Spatial Understanding
• Geometric Context Transformer for Streaming 3D Reconstruction
• ZipMap: Linear-Time Stateful 3D Reconstruction via Test-Time Training
• Flow4R: Unifying 4D Reconstruction and Tracking with Scene Flow
• Chorus: Multi-Teacher Pretraining for Holistic 3D Gaussian Scene Encoding
• From Indoor to Open World: Revealing the Spatial Reasoning Gap in MLLMs
• G3Splat: Geometrically Consistent Generalizable Gaussian Splatting
• FLEG: Feed-Forward Language Embedded Gaussian Splatting from Any Views via Compact Semantic Representation
• 4D-RGPT: Toward Region-level 4D Understanding via Perceptual Distillation
• Scaling Spatial Reasoning in MLLMs through Programmatic Data Synthesis
• In Pursuit of Pixel Supervision for Visual Pre-training
• Multi-View Foundation Models
• Unified Semantic Transformer for 3D Scene Understanding
• I-Scene: 3D Instance Models are Implicit Generalizable Spatial Learners
• Intrinsic Image Fusion for Multi-View 3D Material Reconstruction
• INDOOR-LiDAR: Bridging Simulation and Reality for Robot-Centric 360 degree Indoor LiDAR Perception -- A Robot-Centric Hybrid Dataset
• Audio-Visual Camera Pose Estimation with Passive Scene Sounds and In-the-Wild Video
• E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training
• Any4D: Unified Feed-Forward Metric 4D Reconstruction
• MMSI-Video-Bench: A Holistic Benchmark for Video-Based Spatial Intelligence
• Sharp Monocular View Synthesis in Less Than a Second
• StereoSpace: Depth-Free Synthesis of Stereo Geometry via End-to-End Diffusion in a Canonical Space
• FUSER: Feed-Forward MUltiview 3D Registration Transformer and SE(3)^N Diffusion Refinement
• ASSIST-3D: Adapted Scene Synthesis for Class-Agnostic 3D Instance Segmentation
• Efficiently Reconstructing Dynamic Scenes One D4RT at a Time
• Multi-view Pyramid Transformer: Look Coarser to See Broader
• Hierarchical Image-Guided 3D Point Cloud Segmentation in Industrial Scenes via Multi-View Bayesian Fusion
• LiteVGGT: Boosting Vanilla VGGT via Geometry-aware Cached Token Merging
• Towards Cross-View Point Correspondence in Vision-Language Models
• What Is The Best 3D Scene Representation for Robotics? From Geometric to Foundation Models
• MVRoom: Controllable 3D Indoor Scene Generation with Multi-View Diffusion Models
• Seeing through Imagination: Learning Scene Geometry via Implicit Spatial World Modeling
• Video4Spatial: Towards Visuospatial Intelligence with Context-Guided Video Generation
• SurfFill: Completion of LiDAR Point Clouds via Gaussian Surfel Splatting
• Register Any Point: Scaling 3D Point Cloud Registration by Flow Matching
• SpaceMind: Camera-Guided Modality Fusion for Spatial Reasoning in Vision-Language Models
• Fin3R: Fine-tuning Feed-forward 3D Reconstruction Models via Monocular Knowledge Distillation
• G²VLM: Geometry Grounded Vision Language Model with Unified 3D Reconstruction and Spatial Reasoning
• MODEST: Multi-Optics Depth-of-Field Stereo Dataset
• Vision-Language Memory for Spatial Reasoning
• Wanderland: Geometrically Grounded Simulation for Open-World Embodied AI
• Zoo3D: Zero-Shot 3D Object Detection at Scene Level
• AMB3R: Accurate Feed-forward Metric-scale 3D Reconstruction with Backend
• RADSeg: Unleashing Parameter and Compute Efficient Zero-Shot Open-Vocabulary Segmentation Using Agglomerative Models
• LAST: LeArning to Think in Space and Time for Generalist Vision-Language Models
• Disc3D: Automatic Curation of High-Quality 3D Dialog Data via Discriminative Object Referring
• SegSplat: Feed-forward Gaussian Splatting and Open-Set Semantic Segmentation
• Muskie: Multi-view Masked Image Modeling for 3D Vision Pre-training
• SPIDER: Spatial Image CorresponDence Estimator for Robust Calibration
• CRISTAL: Real-time Camera Registration in Static LiDAR Scans using Neural Rendering
• RoMa v2: Harder Better Faster Denser Feature Matching
• Training-Free Multi-View Extension of IC-Light for Textual Position-Aware Scene Relighting
• Video Spatial Reasoning with Object-Centric 3D Rollout
• Large Language Models and 3D Vision for Intelligent Robotic Perception and Autonomy
• AHA! Animating Human Avatars in Diverse Scenes with Gaussian Splatting
• Wid3R: Wide Field-of-View 3D Reconstruction via Camera Model Conditioning
• YoNoSplat: You Only Need One Model for Feedforward 3D Gaussian Splatting
• How Many Tokens Do 3D Point Cloud Transformer Architectures Really Need?
• Visual Spatial Tuning
• Cambrian-S: Towards Spatial Supersensing in Video
• Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcuts
• Room Envelopes: A Synthetic Dataset for Indoor Layout Reconstruction from Images
• Can Foundation Models Revolutionize Mobile AR Sparse Sensing?
• Class-agnostic 3D Segmentation by Granularity-Consistent Automatic 2D Mask Tracking
• Diff4Splat: Controllable 4D Scene Generation with Latent Dynamic Reconstruction Models
• MoRE: 3D Visual Geometry Reconstruction Meets Mixture-of-Experts
• Glob3R: Global Structure-from-Motion with 3D Foundation Models
• CinemaTraj: Composing Atomic Camera Trajectories for 3D Scenes with LLM Agents
• Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoning
• NOVA3R: Non-pixel-aligned Visual Transformer for Amodal 3D Reconstruction
• AtlasGS: Atlanta-world Guided Surface Reconstruction with Implicit Structured Gaussians
• Generative View Stitching
• NVSim: Novel View Synthesis Simulator for Large Scale Indoor Navigation
• PlanarGS: High-Fidelity Indoor 3D Gaussian Splatting Guided by Vision-Language Planar Priors
• Concerto: Joint 2D-3D Self-Supervised Learning Emerges Spatial Representations
• IGGT: Instance-Grounded Geometry Transformer for Semantic 3D Reconstruction
• OpenHype: Hyperbolic Embeddings for Hierarchical Open-Vocabulary Radiance Fields
• PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting
• SaLon3R: Structure-aware Long-term Generalizable 3D Reconstruction from Unposed Images
• DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoning
• Reasoning in Space via Grounding in the World
• Scene Coordinate Reconstruction Priors
• G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior
• IL3D: A Large-Scale Indoor Layout Dataset for LLM-Driven 3D Scene Generation
• ACE-G: Improving Generalization of Scene Coordinate Regression Through Query Pre-Training
• SNAP: Towards Segmenting Anything in Any Point Cloud
• REACT3D: Recovering Articulations for Interactive Physical 3D Scenes
• Vlaser: Vision-Language-Action Model with Synergistic Embodied Reasoning
• WorldMirror: Universal 3D World Reconstruction with Any-Prior Prompting
• Geometry-Aware Scene Configurations for Novel View Synthesis
• SpaceVista: All-Scale Visual Spatial Reasoning from mm to km
• SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models
• ARTDECO: Towards Efficient and High-Fidelity On-the-Fly 3D Reconstruction with Structured Scene Representation
• RTGS: Real-Time 3D Gaussian Splatting SLAM via Multi-Level Redundancy Reduction
• HoloScene: Simulation-Ready Interactive 3D Worlds from a Single Video
• SegMASt3R: Geometry Grounded Segment Matching
• A Scene is Worth a Thousand Features: Feed-Forward Camera Localization from a Collection of Image Features
• Social 3D Scene Graphs: Modeling Human Actions and Relations for Interactive Service Robots
• ExGS: Extreme 3D Gaussian Compression with Diffusion Priors
• OMeGa: Joint Optimization of Explicit Meshes and Gaussian Splats for Robust Scene-Level Surface Reconstruction
• PROFusion: Robust and Accurate Dense Reconstruction via Camera Pose Regression and Optimization
• DepthLM: Metric Depth From Vision Language Models
• Euclid's Gift: Enhancing Spatial Perception and Reasoning in Vision-Language Models via Geometric Surrogate Tasks
• GeLoc3r: Enhancing Relative Camera Pose Regression with Geometric Consistency Regularization
• Articulated Object Reconstruction from Rest-State Observation
• LoGeR: Long-Context Geometric Reconstruction with Hybrid Memory
• DAGE: Dual-Stream Architecture for Efficient and Fine-Grained Geometry Estimation
• SAIL-Recon: Large SfM by Augmenting Scene Regression with Localization
• SD-VLM: Spatial Measuring and Understanding with Depth-Encoded Vision-Language Models
• SPFSplatV2: Efficient Self-Supervised Pose-Free 3D Gaussian Splatting from Sparse Views
• SLAM-Former: Putting SLAM into One Transformer
• No Need for Real 3D: Fusing 2D Vision with Pseudo 3D Representations for Robotic Manipulation Learning
• See&Trek: Training-Free Spatial Prompting for Multimodal Large Language Model
• 3D Gaussian Flats: Hybrid 2D/3D Photometric Scene Reconstruction
• SPATIALGEN: Layout-guided 3D Indoor Scene Generation
• OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
• Gaussian-Plus-SDF SLAM: High-fidelity 3D Reconstruction at 150+ fps
• 3DAeroRelief: The first 3D Benchmark UAV Dataset for Post-Disaster Assessment
• InternScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layouts
• Fiducial Marker Splatting for High-Fidelity Robotics Simulations
• WinT3R: Window-Based Streaming Reconstruction with Camera Token Pool
• SGS-3D: High-Fidelity 3D Instance Segmentation via Reliable Semantic Mask Splitting and Growing
• ViSTA-SLAM: Visual SLAM with Symmetric Two-view Association
• Robix: A Unified Model for Robot Interaction, Reasoning and Planning
• AGS: Accelerating 3D Gaussian Splatting SLAM via CODEC-Assisted Frame Covisibility Detection
• SceneGen: Single-Image 3D Scene Generation in One Feedforward Pass
• GSFix3D: Diffusion-Guided Repair of Novel Views in Gaussian Splatting
• Tinker: Diffusion's Gift to 3D--Multi-View Consistent Editing From Sparse Inputs without Per-Scene Optimization
• UniUGG: Unified 3D Understanding and Generation via Geometric-Semantic Encoding
• G-CUT3R: Guided 3D Reconstruction with Camera and Depth Prior Integration
• STream3R: Scalable Sequential 3D Reconstruction with Causal Transformer
• A Survey on 3D Gaussian Splatting Applications: Segmentation, Editing, and Generation
• RayletDF: Raylet Distance Fields for Generalizable 3D Surface Reconstruction from Point Clouds or Gaussians
• Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens
• PixCuboid: Room Layout Estimation from Multi-view Featuremetric Alignment
• Trace3D: Consistent Segmentation Lifting via Gaussian Instance Tracing
• Uni3R: Unified 3D Reconstruction and Semantic Understanding via Generalizable Gaussian Splatting from Unposed Multi-View Images
• GaussianCross: Cross-modal Self-supervised 3D Representation Learning via Gaussian Splatting
• Multimodal Referring Segmentation: A Survey
• Details Matter for Indoor Open-vocabulary 3D Instance Segmentation
• UAVScenes: A Multi-Modal Dataset for UAVs
• Ov3R: Open-Vocabulary Semantic 3D Reconstruction from RGB Videos
• S³LAM: Surfel Splatting SLAM for Geometrically Accurate Tracking and Mapping

Related

• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• SceneNet: Understanding Real World Indoor Scenes With Synthetic Data
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding
• InteriorNet: Mega-scale Multi-sensor Photo-realistic Indoor Scenes Dataset