UA-Pose: Uncertainty-Aware 6D Object Pose Estimation and Online Object Completion with Partial References

2025/06/09 by Mingfeng Li, Li, Ming-Feng, Xin Yang +13 · 3 citations
Computer Science · Engineering · #3D pose estimation #Articulated body pose estimation #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generalizability theory #Human Pose and Action Recognition #Multimodal Machine Learning Applications #Object (grammar) #Object detection #Pose #Robot Manipulation and Learning #Robotics (cs.RO) #Robustness (evolution) #Set (abstract data type)

paper · pdf · doi:10.48550/arxiv.2506.07996

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/06/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

6D object pose estimation has shown strong generalizability to novel objects. However, existing methods often require either a complete, well-reconstructed 3D model or numerous reference images that fully cover the object. Estimating 6D poses from partial references, which capture only fragments of an object's appearance and geometry, remains challenging. To address this, we propose UA-Pose, an uncertainty-aware approach for 6D object pose estimation and online object completion specifically designed for partial references. We assume access to either (1) a limited set of RGBD images with known poses or (2) a single 2D image. For the first case, we initialize a partial object 3D model based on the provided images and poses, while for the second, we use image-to-3D techniques to generate an initial object 3D model. Our method integrates uncertainty into the incomplete 3D model, distinguishing between seen and unseen regions. This uncertainty enables confidence assessment in pose estimation and guides an uncertainty-aware sampling strategy for online object completion, enhancing robustness in pose estimation accuracy and improving object completeness. We evaluate our method on the YCB-Video, YCBInEOAT, and HO3D datasets, including RGBD sequences of YCB objects manipulated by robots and human hands. Experimental results demonstrate significant performance improvements over existing methods, particularly when object observations are incomplete or partially captured. Project page: https://minfenli.github.io/UA-Pose/

Citations

• InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models
• SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion
• MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Images
• FoundationPose-TensorRT
• FreeZe: Training-free zero-shot 6D pose estimation with geometric and vision foundation models
• FoundPose: Unseen Object Pose Estimation with Foundation Features
• SAM-6D: Segment Anything Model Meets Zero-Shot 6D Object Pose Estimation
• GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence
• Wonder3D: Single Image to 3D using Cross-Domain Diffusion
• Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model
• ZS6D: Zero-shot 6D Object Pose Estimation using Vision Transformers
• Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors
• ZeroPose: CAD-Prompted Zero-shot Object 6D Pose Estimation in Cluttered Scenes
• Track Anything: Segment Anything Meets Videos
• TTA-COPE: Test-Time Adaptation for Category-Level Object Pose Estimation
• HS-Pose: Hybrid Scope Feature Extraction for Category-level Object Pose Estimation
• BundleSDF: Neural 6-DoF Tracking and 3D Reconstruction of Unknown Objects
• Zero-1-to-3: Zero-shot One Image to 3D Object
• OnePose++: Keypoint-Free One-Shot Object Pose Estimation without CAD Models
• MegaPose: 6D Pose Estimation of Novel Objects via Render & Compare
• SSP-Pose: Symmetry-Aware Shape Prior Deformation for Direct Category-Level Object Pose Estimation
• OnePose: One-Shot Object Pose Estimation without CAD Models
• Gen6D: Generalizable Model-Free 6-DoF Object Pose Estimation from RGB Images
• OSOP: A Multi-Stage One Shot Object Pose Estimation Framework
• FS6D: Few-Shot 6D Pose Estimation of Novel Objects
• Instant Neural Graphics Primitives with a Multiresolution Hash Encoding
• NICE-SLAM: Neural Implicit Scalable Encoding for SLAM
• CaTGrasp: Learning Category-Level Task-Relevant Grasping in Clutter from Simulation
• BundleTrack: 6D Pose Tracking for Novel Objects without Instance or Category-Level 3D Models
• NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
• LoFTR: Detector-Free Local Feature Matching with Transformers
• FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimation
• Modular Primitives for High-Performance Differentiable Rendering
• CosyPose: Consistent multi-view multi-object 6D pose estimation
• Shape Prior Deformation for Categorical 6D Object Pose and Size Estimation
• Implicit Geometric Regularization for Learning Shapes
• Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimation
• Accelerating 3D Deep Learning with PyTorch3D
• PVN3D: A Deep Point-wise 3D Keypoints Voting Network for 6DoF Pose Estimation
• Kornia: an Open Source Differentiable Computer Vision Library for PyTorch
• Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation
• PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes

Cited by

• ConceptPose: Training-Free Zero-Shot Object Pose Estimation using Concept Vectors
• VFM-VLM: Vision Foundation Model and Vision Language Model based Visual Comparison for 3D Pose Estimation
• MAPRPose: Mask-Aware Proposal and Amodal Refinement for Multi-Object 6D Pose Estimation

Related

• Unseen Object 6D Pose Estimation: A Benchmark and Baselines
• Pose Estimation of Specific Rigid Objects
• 6IMPOSE: Bridging the Reality Gap in 6D Pose Estimation for Robotic Grasping
• WALDO: Where Unseen Model-based 6D Pose Estimation Meets Occlusion
• MaskVal: Simple but Effective Uncertainty Quantification for 6D Pose Estimation