2019/12/02 by Aritra Bhowmik, Bhowmik, Aritra, Stefan Gumhold +5 · 2 citations
Computer Science · Engineering · #Advanced Image and Video Retrieval Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Human Pose and Action Recognition #Machine Learning (cs.LG) #Robotics and Sensor-Based Localization
paper · pdf · doi:10.48550/arxiv.1912.00623
openalex publication_date 2019/12/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We address a core problem of computer vision: Detection and description of 2D\nfeature points for image matching. For a long time, hand-crafted designs, like\nthe seminal SIFT algorithm, were unsurpassed in accuracy and efficiency.\nRecently, learned feature detectors emerged that implement detection and\ndescription using neural networks. Training these networks usually resorts to\noptimizing low-level matching scores, often pre-defining sets of image patches\nwhich should or should not match, or which should or should not contain key\npoints. Unfortunately, increased accuracy for these low-level matching scores\ndoes not necessarily translate to better performance in high-level vision\ntasks. We propose a new training methodology which embeds the feature detector\nin a complete vision pipeline, and where the learnable parameters are trained\nin an end-to-end fashion. We overcome the discrete nature of key point\nselection and descriptor matching using principles from reinforcement learning.\nAs an example, we address the task of relative pose estimation between a pair\nof images. We demonstrate that the accuracy of a state-of-the-art\nlearning-based feature detector can be increased when trained for the task it\nis supposed to solve at test time. Our training methodology poses little\nrestrictions on the task to learn, and works for any architecture which\npredicts key point heat maps, and descriptors for key point locations.\n