SegVec3D: A Method for Vector Embedding of 3D Objects Oriented Towards Robot manipulation

2025/07/13 by Boyu Wang, Kang, Zhihan, Wang, Boyu
Engineering · #Manufacturing Process and Optimization

paper · pdf · doi:10.48550/arxiv.2507.09459

Abstract

We propose SegVec3D, a novel framework for 3D point cloud instance segmentation that integrates attention mechanisms, embedding learning, and cross-modal alignment. The approach builds a hierarchical feature extractor to enhance geometric structure modeling and enables unsupervised instance segmentation via contrastive clustering. It further aligns 3D data with natural language queries in a shared semantic space, supporting zero-shot retrieval. Compared to recent methods like Mask3D and ULIP, our method uniquely unifies instance segmentation and multimodal understanding with minimal supervision and practical deployability.

Citations

• Point Transformer V2: Grouped Vector Attention and Partition-based Pooling
• Mask3D: Mask Transformer for 3D Semantic Instance Segmentation
• PointCLIP: Point Cloud Understanding by CLIP
• Neural Fields in Visual Computing and Beyond
• Neural Fields in Visual Computing and Beyond
• Align before Fuse: Vision and Language Representation Learning with Momentum Distillation
• Vector Neurons: A General Framework for SO(3)-Equivariant Networks
• PAConv: Position Adaptive Convolution with Dynamic Kernel Assembling on Point Clouds
• Learning Transferable Visual Models From Natural Language Supervision
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds
• 3D Instance Segmentation via Multi-Task Metric Learning
• Multimodal Transformer for Unaligned Multimodal Language Sequences
• KPConv: Flexible and Deformable Convolution for Point Clouds
• PointConv: Deep Convolutional Networks on 3D Point Clouds
• Dynamic Graph CNN for Learning on Point Clouds
• Dynamic Graph CNN for Learning on Point Clouds
• SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentation
• Graph Attention Networks
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Inductive Representation Learning on Large Graphs
• Multimodal Machine Learning: A Survey and Taxonomy
• Multimodal Machine Learning: A Survey and Taxonomy
• Multi-view Supervision for Single-view Reconstruction via Differentiable\n Ray Consistency
• Joint 2D-3D-Semantic Data for Indoor Scene Understanding
• ShapeNet: An Information-Rich 3D Model Repository

Related

• All in One: Visual-Description-Guided Unified Point Cloud Segmentation
• SegDINO3D: 3D Instance Segmentation Empowered by Both Image-Level and Object-Level 2D Features
• Generalized Robot 3D Vision-Language Model with Fast Rendering and Pre-Training Vision-Language Alignment
• Interactive Object Segmentation in 3D Point Clouds
• Points2Vec: Unsupervised Object-level Feature Learning from Point Clouds