2019/04/22 by Mengdan Feng, Feng, Mengdan, Sixing Hu +5 · 3 citations
Computer Science · Earth and Planetary Sciences · Engineering · #3D Surveying and Cultural Heritage #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Robotics and Sensor-Based Localization
paper · pdf · doi:10.48550/arxiv.1904.09742
openalex publication_date 2019/04/22 · openalex created_date 2019/04/25 · openalex updated_date 2026/07/28
Large-scale point cloud generated from 3D sensors is more accurate than its image-based counterpart. However, it is seldom used in visual pose estimation due to the difficulty in obtaining 2D-3D image to point cloud correspondences. In this paper, we propose the 2D3D-MatchNet - an end-to-end deep network architecture to jointly learn the descriptors for 2D and 3D keypoint from image and point cloud, respectively. As a result, we are able to directly match and establish 2D-3D correspondences from the query image and 3D point cloud reference map for visual pose estimation. We create our Oxford 2D-3D Patches dataset from the Oxford Robotcar dataset with the ground truth camera poses and 2D-3D image to point cloud correspondences for training and testing the deep network. Experimental results verify the feasibility of our approach.