SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation

2023/11/18 by Yamei Chen, Chen, Yamei, Yan Di +15 · 13 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Robot Manipulation and Learning

paper · pdf · doi:10.48550/arxiv.2311.11125

openalex publication_date 2023/11/18 · openalex created_date 2023/11/23 · openalex updated_date 2026/07/28

Abstract

Category-level object pose estimation, aiming to predict the 6D pose and 3D size of objects from known categories, typically struggles with large intra-class shape variation. Existing works utilizing mean shapes often fall short of capturing this variation. To address this issue, we present SecondPose, a novel approach integrating object-specific geometric features with semantic category priors from DINOv2. Leveraging the advantage of DINOv2 in providing SE(3)-consistent semantic features, we hierarchically extract two types of SE(3)-invariant geometric features to further encapsulate local-to-global object-specific information. These geometric features are then point-aligned with DINOv2 features to establish a consistent object representation under SE(3) transformations, facilitating the mapping from camera space to the pre-defined canonical space, thus further enhancing pose estimation. Extensive experiments on NOCS-REAL275 demonstrate that SecondPose achieves a 12.4% leap forward over the state-of-the-art. Moreover, on a more complex dataset HouseCat6D which provides photometrically challenging objects, SecondPose still surpasses other competitors by a large margin.

Cited by

• THE-Pose: Topological Prior with Hybrid Graph Fusion for Estimating Category-Level 6D Object Pose
• Box6D : Zero-shot Category-level 6D Pose Estimation of Warehouse Boxes
• OPFormer: Object Pose Estimation leveraging foundation model with geometric encoding
• RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimation
• Foundation Visual Encoders Are Secretly Few-Shot Anomaly Detectors
• Joint Learning of Pose Regression and Denoising Diffusion with Score Scaling Sampling for Category-level 6D Pose Estimation
• SCOPE: Semantic Conditioning for Sim2Real Category-Level Object Pose Estimation in Robotics
• Category-Level Object Shape and Pose Estimation in Less Than a Millisecond
• UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimation
• Learning Point Cloud Representations with Pose Continuity for Depth-Based Category-Level 6D Object Pose Estimation
• RCGNet: RGB-based Category-Level 6D Object Pose Estimation with Geometric Guidance
• Forecasting Continuous Non-Conservative Dynamical Systems in SO(3)
• Unified Category-Level Object Detection and Pose Estimation from RGB Images using 3D Prototypes

Related

• GS-Pose: Category-Level Object Pose Estimation via Geometric and Semantic Correspondence
• GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting
• Leveraging SE(3) Equivariance for Self-Supervised Category-Level Object Pose Estimation
• IST-Net: Prior-free Category-level Pose Estimation with Implicit Space Transformation
• GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimation