Embodied Active Domain Adaptation for Semantic Segmentation via Informative Path Planning

2022/03/01 by René Zurbrügg, Hermann Blum, Zurbrügg, René +7 · 2 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Human Pose and Action Recognition #Multimodal Machine Learning Applications #Robotics (cs.RO)

paper · pdf · doi:10.48550/arxiv.2203.00549

openalex publication_date 2022/03/01 · openalex created_date 2022/04/03 · openalex updated_date 2026/07/28

Abstract

This work presents an embodied agent that can adapt its semantic segmentation network to new indoor environments in a fully autonomous way. Because semantic segmentation networks fail to generalize well to unseen environments, the agent collects images of the new environment which are then used for self-supervised domain adaptation. We formulate this as an informative path planning problem, and present a novel information gain that leverages uncertainty extracted from the semantic model to safely collect relevant data. As domain adaptation progresses, these uncertainties change over time and the rapid learning feedback of our system drives the agent to collect different data. Experiments show that our method adapts to new environments faster and with higher final performance compared to an exploration objective, and can successfully be deployed to real-world environments on physical robots.

Cited by

• Box Pose and Shape Estimation and Domain Adaptation for Large-Scale Warehouse Automation
• Semantics-aware Predictive Inspection Path Planning

Related

• Semi-Supervised Active Learning for Semantic Segmentation in Unknown Environments Using Informative Path Planning
• Embodied Visual Active Learning for Semantic Segmentation
• SegmATRon: Embodied Adaptive Semantic Segmentation for Indoor Environment
• Self-supervised Pre-training for Semantic Segmentation in an Indoor Scene
• D2ADA: Dynamic Density-aware Active Domain Adaptation for Semantic Segmentation