Towards Semantic Segmentation of Urban-Scale 3D Point Clouds: A Dataset, Benchmarks and Challenges

2020/09/07 by Qingyong Hu, Hu, Qingyong, Bo Yang +9 · 5 citations
Earth and Planetary Sciences · Engineering · Environmental Science · #3D Shape Modeling and Analysis #3D Surveying and Cultural Heritage #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Remote Sensing and LiDAR Applications #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2009.03137

openalex publication_date 2020/09/07 · openalex created_date 2020/09/11 · openalex updated_date 2026/07/28

Abstract

An essential prerequisite for unleashing the potential of supervised deep learning algorithms in the area of 3D scene understanding is the availability of large-scale and richly annotated datasets. However, publicly available datasets are either in relative small spatial scales or have limited semantic annotations due to the expensive cost of data acquisition and data annotation, which severely limits the development of fine-grained semantic understanding in the context of 3D point clouds. In this paper, we present an urban-scale photogrammetric point cloud dataset with nearly three billion richly annotated points, which is three times the number of labeled points than the existing largest photogrammetric point cloud dataset. Our dataset consists of large areas from three UK cities, covering about 7.6 km² of the city landscape. In the dataset, each 3D point is labeled as one of 13 semantic classes. We extensively evaluate the performance of state-of-the-art algorithms on our dataset and provide a comprehensive analysis of the results. In particular, we identify several key challenges towards urban-scale point cloud understanding. The dataset is available at https://github.com/QingyongHu/SensatUrban.

Citations

• ShapeNet: An Information-Rich 3D Model Repository
• SceneNet: Understanding Real World Indoor Scenes With Synthetic Data
• A scalable active framework for region annotation in 3D shape collections
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Paris-Lille-3D: a large and high-quality ground truth urban point cloud dataset for automatic segmentation and classification
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• A2D2: Audi Autonomous Driving Dataset
• Scalability in Perception for Autonomous Driving: Waymo Open Dataset
• Deep Learning for 3D Point Clouds: A Survey

Cited by

• SemanticBridge - A Dataset for 3D Semantic Segmentation of Bridges and Domain Gap Analysis
• OpenUrban3D: Annotation-Free Open-Vocabulary Semantic Segmentation of Large-Scale Urban Point Clouds
• Towards scalable organ level 3D plant segmentation: Bridging the data algorithm computing gap
• EarthCrafter: Scalable 3D Earth Generation via Dual-Sparse Latent Diffusion
• Unsupervised Point Cloud Pre-Training via Occlusion Completion

Related

• SensatUrban: Learning Semantics from Urban-Scale Photogrammetric Point Clouds
• Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark
• Semantic Segmentation on Swiss3DCities: A Benchmark Study on Aerial Photogrammetric 3D Pointcloud Dataset
• Deep learning for 3D point cloud processing -- from approaches, tasks to its implications on urban and environmental applications
• Efficient Urban-scale Point Clouds Segmentation with BEV Projection