SurfEmb: Dense and Continuous Correspondence Distributions for Object Pose Estimation with Learnt Surface Embeddings

2021/11/26 by Rasmus Laurvig Haugaard, Haugaard, Rasmus Laurvig, Anders Glent Buch +1 · 11 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Robot Manipulation and Learning #Robotics (cs.RO) #Robotics and Sensor-Based Localization

paper · pdf · doi:10.48550/arxiv.2111.13489

openalex publication_date 2021/11/26 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We present an approach to learn dense, continuous 2D-3D correspondence distributions over the surface of objects from data with no prior knowledge of visual ambiguities like symmetry. We also present a new method for 6D pose estimation of rigid objects using the learnt distributions to sample, score and refine pose hypotheses. The correspondence distributions are learnt with a contrastive loss, represented in object-specific latent spaces by an encoder-decoder query model and a small fully connected key model. Our method is unsupervised with respect to visual ambiguities, yet we show that the query- and key models learn to represent accurate multi-modal surface distributions. Our pose estimation method improves the state-of-the-art significantly on the comprehensive BOP Challenge, trained purely on synthetic data, even compared with methods trained on real data. The project site is at https://surfemb.github.io/ .

Cited by

• Learning Dense 2D-3D Correspondence for X-ray-to-CT Registration of Knee Bones
• THE-Pose: Topological Prior with Hybrid Graph Fusion for Estimating Category-Level 6D Object Pose
• VFM-VLM: Vision Foundation Model and Vision Language Model based Visual Comparison for 3D Pose Estimation
• Object Pose Distribution Estimation for Determining Revolution and Reflection Uncertainty in Point Clouds
• GNC-Pose: Geometry-Aware GNC-PnP for Accurate 6D Pose Estimation
• Is Image-based Object Pose Estimation Ready to Support Grasping?
• Uncertainty Quantification for Visual Object Pose Estimation
• WALDO: Where Unseen Model-based 6D Pose Estimation Meets Occlusion
• OPFormer: Object Pose Estimation leveraging foundation model with geometric encoding
• RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimation
• Generative Visual Foresight Meets Task-Agnostic Pose Estimation in Robotic Table-Top Manipulation

Related

• Continuous Surface Embeddings
• Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wild
• Scene-level Pose Estimation for Multiple Instances of Densely Packed Objects
• Multi-view object pose estimation from correspondence distributions and epipolar geometry
• NeuSurfEmb: A Complete Pipeline for Dense Correspondence-based 6D Object Pose Estimation without CAD Models