Learning Dense Visual Descriptors using Image Augmentations for Robot Manipulation Tasks

2022/09/12 by Christian Gräf, Graf, Christian, David B. Adrian +13 · 2 citations
Engineering · #Robot Manipulation and Learning #Robotics and Sensor-Based Localization #Image Processing Techniques and Applications

paper · pdf · doi:10.48550/arxiv.2209.05213

Abstract

We propose a self-supervised training approach for learning view-invariant dense visual descriptors using image augmentations. Unlike existing works, which often require complex datasets, such as registered RGBD sequences, we train on an unordered set of RGB images. This allows for learning from a single camera view, e.g., in an existing robotic cell with a fix-mounted camera. We create synthetic views and dense pixel correspondences using data augmentations. We find our descriptors are competitive to the existing methods, despite the simpler data recording and setup requirements. We show that training on synthetic correspondences provides descriptor consistency across a broad range of camera views. We compare against training with geometric correspondence from multiple views and provide ablation studies. We also show a robotic bin-picking experiment using descriptors learned from a fix-mounted camera for defining grasp preferences.

Cited by

• ManiLong-Shot: Interaction-Aware One-Shot Imitation Learning for Long-Horizon Manipulation
• M³A Policy: Mutable Material Manipulation Augmentation Policy through Photometric Re-rendering

Related

• Dense Object Nets: Learning Dense Visual Object Descriptors By and For Robotic Manipulation
• Cycle-Correspondence Loss: Learning Dense View-Invariant Visual Features from Unlabeled and Unordered RGB Images
• Supervised Training of Dense Object Nets using Optimal Descriptors for\n Industrial Robotic Applications
• Efficient and Robust Training of Dense Object Nets for Multi-Object Robot Manipulation
• NeRF-Supervision: Learning Dense Object Descriptors from Neural Radiance Fields