2023/10/02 by Advaith V. Sethuraman, Sethuraman, Advaith Venkatramanan, Katherine A. Skinner +1 · 1 citation
Arts and Humanities · Computer Science · Earth and Planetary Sciences · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Maritime and Coastal Archaeology #Underwater Acoustics Research
paper · pdf · doi:10.48550/arxiv.2310.01667
openalex publication_date 2023/10/02 · openalex created_date 2023/10/05 · openalex updated_date 2026/07/28
In this paper, we address the problem of sim-to-real transfer for object segmentation when there is no access to real examples of an object of interest during training, i.e. zero-shot sim-to-real transfer for segmentation. We focus on the application of shipwreck segmentation in side scan sonar imagery. Our novel segmentation network, STARS, addresses this challenge by fusing a predicted deformation field and anomaly volume, allowing it to generalize better to real sonar images and achieve more effective zero-shot sim-to-real transfer for image segmentation. We evaluate the sim-to-real transfer capabilities of our method on a real, expert-labeled side scan sonar dataset of shipwrecks collected from field work surveys with an autonomous underwater vehicle (AUV). STARS is trained entirely in simulation and performs zero-shot shipwreck segmentation with no additional fine-tuning on real data. Our method provides a significant 20% increase in segmentation performance for the targeted shipwreck class compared to the best baseline.