Joint 2D-3D-Semantic Data for Indoor Scene Understanding

2017/02/03 by Iro Armeni, Armeni, Iro, Sasha Sax +5 · 41 citations
Earth and Planetary Sciences · Engineering · Environmental Science · #3D Surveying and Cultural Heritage #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Remote Sensing and LiDAR Applications #Robotics (cs.RO) #Robotics and Sensor-Based Localization

paper · pdf · doi:10.48550/arxiv.1702.01105

openalex publication_date 2017/02/03 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present a dataset of large-scale indoor spaces that provides a variety of mutually registered modalities from 2D, 2.5D and 3D domains, with instance-level semantic and geometric annotations. The dataset covers over 6,000m2 and contains over 70,000 RGB images, along with the corresponding depths, surface normals, semantic annotations, global XYZ images (all in forms of both regular and 360° equirectangular images) as well as camera information. It also includes registered raw and semantically annotated 3D meshes and point clouds. The dataset enables development of joint and cross-modal learning models and potentially unsupervised approaches utilizing the regularities present in large-scale indoor spaces. The dataset is available here: http://3Dsemantics.stanford.edu/

Citations

• ShapeNet: An Information-Rich 3D Model Repository

Cited by

• Depth Anything in 360^∘: Towards Scale Invariance in the Wild
• Depth Any Panoramas: A Foundation Model for Panoramic Depth Estimation
• Power of Boundary and Reflection: Semantic Transparent Object Segmentation using Pyramid Vision Transformer with Transparent Cues
• AirSim360: A Panoramic Simulation Platform within Drone View
• C3Po: Cross-View Cross-Modality Correspondence by Pointmap Prediction
• Unified Spherical Frontend: Learning Rotation-Equivariant Representations of Spherical Images from Any Camera
• EPSegFZ: Efficient Point Cloud Semantic Segmentation for Few- and Zero-Shot Scenarios with Language Guidance
• Wid3R: Wide Field-of-View 3D Reconstruction via Camera Model Conditioning
• Genie Sim PanoWorld: An Infinite Indoor 3D World Generation Pipeline via Panoramic Scene Modeling and Simulation
• Full segmentation annotations of 3D time-lapse microscopy images of MDA231 cells
• Thinking with Camera: A Unified Multimodal Model for Camera-Centric Understanding and Generation
• DA²: Depth Anything in Any Direction
• RPG360: Robust 360 Depth Estimation with Perspective Foundation Models and Graph Optimization
• FastViDAR: Real-Time Omnidirectional Depth Estimation via Alternative Hierarchical Attention
• Manhattan Room Layout Reconstruction from a Single 360 image: A Comparative Study of State-of-the-art Methods
• 3DContextNet: K-d Tree Guided Hierarchical Learning of Point Clouds Using Local and Global Contextual Cues
• Survey on semantic segmentation using deep learning techniques
• TextureNet: Consistent Local Parametrizations for Learning from High-Resolution Signals on Meshes
• PANORAMA: The Rise of Omnidirectional Vision in the Embodied AI Era
• PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers
• PointAR: Efficient Lighting Estimation for Mobile Augmented Reality
• SoundSpaces: Audio-Visual Navigation in 3D Environments
• A Closer Look at Local Aggregation Operators in Point Cloud Analysis
• Deep Visual MPC-Policy Learning for Navigation
• Towards Real-Time Monocular Depth Estimation for Robotics: A Survey
• Integrating SAM Supervision for 3D Weakly Supervised Point Cloud Segmentation
• Weakly Supervised Silhouette-based Semantic Scene Change Detection
• MASC: Multi-scale Affinity with Sparse Convolution for 3D Instance Segmentation
• SpherePHD: Applying CNNs on a Spherical PolyHeDron Representation of 360\n degree Images
• HorizonNet: Learning Room Layout with 1D Representation and Pano Stretch Data Augmentation
• Indoor Panorama Planar 3D Reconstruction via Divide and Conquer
• Enhancing Construction Site Analysis and Understanding with 3D Segmentation
• Propagating Sparse Depth via Depth Foundation Model for Out-of-Distribution Depth Completion
• PixCuboid: Room Layout Estimation from Multi-view Featuremetric Alignment
• Glass Surface Segmentation with an RGB-D Camera via Weighted Feature Fusion for Service Robots
• Cross-Dataset Semantic Segmentation Performance Analysis: Unifying NIST Point Cloud City Datasets for 3D Deep Learning
• Semantic Segmentation for Real Point Cloud Scenes via Bilateral Augmentation and Adaptive Fusion
• Multi-view PointNet for 3D Scene Understanding
• UAVScenes: A Multi-Modal Dataset for UAVs
• One Thing One Click: A Self-Training Approach for Weakly Supervised 3D Semantic Segmentation
• End-to-end learning of keypoint detection and matching for relative pose estimation

Related

• SPATIALGEN: Layout-guided 3D Indoor Scene Generation
• Matterport3D: Learning from RGB-D Data in Indoor Environments
• IL3D: A Large-Scale Indoor Layout Dataset for LLM-Driven 3D Scene Generation
• High-quality Instance-aware Semantic 3D Map Using RGB-D Camera
• SSR-2D: Semantic 3D Scene Reconstruction from 2D Images