2022/06/24 by Peng Jiang, Srikanth Saripalli, Jiang, Peng +1 · 1 citation
Earth and Planetary Sciences · Engineering · Environmental Science · #3D Shape Modeling and Analysis #3D Surveying and Cultural Heritage #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Remote Sensing and LiDAR Applications
paper · pdf · doi:10.48550/arxiv.2206.12071
openalex publication_date 2022/06/24 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Image and Point Clouds provide different information for robots. Finding the correspondences between data from different sensors is crucial for various tasks such as localization, mapping, and navigation. Learning-based descriptors have been developed for single sensors; there is little work on cross-modal features. This work treats learning cross-modal features as a dense contrastive learning problem. We propose a Tuple-Circle loss function for cross-modality feature learning. Furthermore, to learn good features and not lose generality, we developed a variant of widely used PointNet++ architecture for point cloud and U-Net CNN architecture for images. Moreover, we conduct experiments on a real-world dataset to show the effectiveness of our loss function and network structure. We show that our models indeed learn information from both images as well as LiDAR by visualizing the features.