Point2Sequence: Learning the Shape Representation of 3D Point Clouds with an Attention-based Sequence to Sequence Network

2018/11/06 by Xinhai Liu, Zhizhong Han, Liu, Xinhai +5 · 10 citations
Engineering · Computer Science · Earth and Planetary Sciences · #3D Shape Modeling and Analysis #Image Processing and 3D Reconstruction #3D Surveying and Cultural Heritage

paper · pdf · doi:10.48550/arxiv.1811.02565

Abstract

Exploring contextual information in the local region is important for shape understanding and analysis. Existing studies often employ hand-crafted or explicit ways to encode contextual information of local regions. However, it is hard to capture fine-grained contextual information in hand-crafted or explicit manners, such as the correlation between different areas in a local region, which limits the discriminative ability of learned features. To resolve this issue, we propose a novel deep learning model for 3D point clouds, named Point2Sequence, to learn 3D shape features by capturing fine-grained contextual information in a novel implicit way. Point2Sequence employs a novel sequence learning model for point clouds to capture the correlations by aggregating multi-scale areas of each local region with attention. Specifically, Point2Sequence first learns the feature of each area scale in a local region. Then, it captures the correlation between area scales in the process of aggregating all area scales using a recurrent neural network (RNN) based encoder-decoder structure, where an attention mechanism is proposed to highlight the importance of different area scales. Experimental results show that Point2Sequence achieves state-of-the-art performance in shape classification and segmentation tasks.

Cited by

• Fast Hierarchical Neural Network for Feature Learning on Point Cloud
• MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference
• TACO-Net: Topological Signatures Triumph in 3D Object Classification
• A Sketch Based 3D Shape Retrieval Approach Based on Efficient Deep Point-to-Subspace Metric Learning
• VesselRW: Weakly Supervised Subcutaneous Vessel Segmentation via Learned Random Walk Propagation
• DualResolution Residual Architecture with Artifact Suppression for Melanocytic Lesion Segmentation
• Edge Detection for Organ Boundaries via Top Down Refinement and SubPixel Upsampling
• Deep Learning Based Dynamic Environment Reconstruction for Vehicular ISAC Scenarios
• Deeply Dual Supervised learning for melanoma recognition
• Describe, Adapt and Combine: Empowering CLIP Encoders for Open-set 3D Object Retrieval

Related

• LRC-Net: Learning Discriminative Features on Point Clouds by Encoding Local Region Contexts
• Point Attention Network for Semantic Segmentation of 3D Point Clouds
• Cross-Shape Attention for Part Segmentation of 3D Point Clouds
• PointAttN: You Only Need Attention for Point Cloud Completion
• GAPNet: Graph Attention based Point Neural Network for Exploiting Local Feature of Point Cloud