PU-Transformer: Point Cloud Upsampling Transformer

2021/11/24 by Shi Qiu, Saeed Anwar, Qiu, Shi +3 · 5 citations
Earth and Planetary Sciences · Engineering · Environmental Science · #3D Shape Modeling and Analysis #3D Surveying and Cultural Heritage #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Remote Sensing and LiDAR Applications

paper · pdf · doi:10.48550/arxiv.2111.12242

openalex publication_date 2021/11/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Given the rapid development of 3D scanners, point clouds are becoming popular in AI-driven machines. However, point cloud data is inherently sparse and irregular, causing significant difficulties for machine perception. In this work, we focus on the point cloud upsampling task that intends to generate dense high-fidelity point clouds from sparse input data. Specifically, to activate the transformer's strong capability in representing features, we develop a new variant of a multi-head self-attention structure to enhance both point-wise and channel-wise relations of the feature map. In addition, we leverage a positional fusion block to comprehensively capture the local context of point cloud data, providing more position-related information about the scattered points. As the first transformer model introduced for point cloud upsampling, we demonstrate the outstanding performance of our approach by comparing with the state-of-the-art CNN-based methods on different benchmarks quantitatively and qualitatively.

Citations

• ShapeNet: An Information-Rich 3D Model Repository
• ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
• Deep Learning for 3D Point Clouds: A Survey
• Deformable DETR: Deformable Transformers for End-to-End Object Detection
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Transformers in Vision: A Survey
• Pre-Trained Image Processing Transformer
• PnP-3D: A Plug-and-Play for 3D Point Clouds
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Cited by

• PUFM++: Point Cloud Upsampling via Enhanced Flow Matching
• PointNSP: Autoregressive 3D Point Cloud Generation with Next-Scale Level-of-Detail Prediction
• PU-Gaussian: Point Cloud Upsampling using 3D Gaussian Representation
• Point upsampling networks for single-photon sensing
• Preserving Topological and Geometric Embeddings for Point Cloud Recovery

Related

• Transformers in 3D Point Clouds: A Survey
• Point Cloud Recognition with Position-to-Structure Attention Transformers
• Applying Plain Transformers to Real-World Point Clouds
• Dual Transformer for Point Cloud Analysis
• PVT: Point-Voxel Transformer for Point Cloud Learning