CNOS: A Strong Baseline for CAD-based Novel Object Segmentation

2023/07/20 by Van Nguyen Nguyen, Nguyen, Van Nguyen, Thibault Groueix +7 · 16 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Robotics and Sensor-Based Localization #Visual Attention and Saliency Detection

paper · pdf · doi:10.48550/arxiv.2307.11067

Abstract

We propose a simple three-stage approach to segment unseen objects in RGB images using their CAD models. Leveraging recent powerful foundation models, DINOv2 and Segment Anything, we create descriptors and generate proposals, including binary masks for a given input RGB image. By matching proposals with reference descriptors created from CAD models, we achieve precise object ID assignment along with modal masks. We experimentally demonstrate that our method achieves state-of-the-art results in CAD-based novel object segmentation, surpassing existing approaches on the seven core datasets of the BOP challenge by 19.8% AP using the same BOP evaluation protocol. Our source code is available at https://github.com/nv-nguyen/cnos.

Cited by

• Robust 6-DoF Object Pose Tracking with Built-In Recovery under Occlusions and Rapid Object Motions
• RRTrack: Robust and Recoverable Object 6D Pose Tracking for Dynamic Scenes
• AlignPose: Generalizable 6D Pose Estimation via Multi-view Feature-metric Alignment
• Geo6DPose: Fast Zero-Shot 6D Object Pose Estimation via Geometry-Filtered Feature Matching
• PoseGAM: Robust Unseen Object Pose Estimation via Geometry-Aware Multi-View Reasoning
• See in Depth: Training-Free Surgical Scene Segmentation with Monocular Depth Priors
• WALDO: Where Unseen Model-based 6D Pose Estimation Meets Occlusion
• OPFormer: Object Pose Estimation leveraging foundation model with geometric encoding
• STORM: Segment, Track, and Object Re-Localization from a Single Image
• SE(3)-PoseFlow: Estimating 6D Pose Distributions for Uncertainty-Aware Robotic Manipulation
• BioDet: Boosting Industrial Object Detection with Image Preprocessing Strategies
• RayPose: Ray Bundling Diffusion for Template Views in Unseen 6D Object Pose Estimation
• MUSE: Model-based Uncertainty-aware Similarity Estimation for zero-shot 2D Object Detection and Segmentation
• GraspFactory: A Large Object-Centric Grasping Dataset
• Self-supervised Learning Of Visual Pose Estimation Without Pose Labels By Classifying LED States
• MR6D: Benchmarking 6D Pose Estimation for Mobile Robots
• Conditional Latent Diffusion Models for Zero-Shot Instance Segmentation

Related

• NOCTIS: Novel Object Cyclic Threshold based Instance Segmentation
• IAM: Enhancing RGB-D Instance Segmentation with New Benchmarks
• Adapting Pre-Trained Vision Models for Novel Instance Detection and Segmentation
• SAD: Segment Any RGBD
• The Best of Both Modes: Separately Leveraging RGB and Depth for Unseen Object Instance Segmentation