Label-Efficient Grasp Joint Prediction with Point-JEPA

2025/09/13 by Jed Guzelkabaagac, Guzelkabaagac, Jed, Petrović, Boris
Engineering · Neuroscience · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Motor Control and Adaptation #Muscle activation and electromyography studies #Robot Manipulation and Learning #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2509.13349

openalex publication_date 2025/09/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We study whether 3D self-supervised pretraining with Point--JEPA enables label-efficient grasp joint-angle prediction. Meshes are sampled to point clouds and tokenized; a ShapeNet-pretrained Point--JEPA encoder feeds a K=5 multi-hypothesis head trained with winner-takes-all and evaluated by top--logit selection. On a multi-finger hand dataset with strict object-level splits, Point--JEPA improves top--logit RMSE and Coverage@15^∘ in low-label regimes (e.g., 26% lower RMSE at 25% data) and reaches parity at full supervision, suggesting JEPA-style pretraining is a practical lever for data-efficient grasp learning.

Citations

• 3D-JEPA: A Joint Embedding Predictive Architecture for 3D Self-Supervised Representation Learning
• Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloud
• Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand
• Point2Vec for Self-Supervised Representation Learning on Point Clouds
• Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
• Masked Autoencoders for Point Cloud Self-supervised Learning
• data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language
• Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling
• Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes
• CoverNet: Multimodal Behavior Prediction using Trajectory Sets
• PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space
• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
• Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses
• High precision grasp pose detection in dense clutter

Related

• KGN-Pro: Keypoint-Based Grasp Prediction through Probabilistic 2D-3D Correspondence Learning
• Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics
• Combining Shape Completion and Grasp Prediction for Fast and Versatile Grasping with a Multi-Fingered Hand
• Leveraging CVAE for Joint Configuration Estimation of Multifingered Grippers from Point Cloud Data
• End-to-End Learning to Grasp via Sampling from Object Point Clouds