Efficient Neighbourhood Consensus Networks via Submanifold Sparse Convolutions

2020/04/22 by Ignacio Rocco, Rocco, Ignacio, Relja Arandjelović +3 · 8 citations
Computer Science · #Advanced Neural Network Applications #Advanced Vision and Imaging #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis

paper · pdf · doi:10.48550/arxiv.2004.10566

openalex publication_date 2020/04/22 · openalex created_date 2020/05/01 · openalex updated_date 2026/07/28

Abstract

In this work we target the problem of estimating accurately localised correspondences between a pair of images. We adopt the recent Neighbourhood Consensus Networks that have demonstrated promising performance for difficult correspondence problems and propose modifications to overcome their main limitations: large memory consumption, large inference time and poorly localised correspondences. Our proposed modifications can reduce the memory footprint and execution time more than 10×, with equivalent results. This is achieved by sparsifying the correlation tensor containing tentative matches, and its subsequent processing with a 4D CNN using submanifold sparse convolutions. Localisation accuracy is significantly improved by processing the input images in higher resolution, which is possible due to the reduced memory footprint, and by a novel two-stage correspondence relocalisation module. The proposed Sparse-NCNet method obtains state-of-the-art results on the HPatches Sequences and InLoc visual localisation benchmarks, and competitive results in the Aachen Day-Night benchmark.

Citations

• Spatially-sparse convolutional neural networks
• Random sample consensus
• Distinctive Image Features from Scale-Invariant Keypoints
• Sparse 3D convolutional neural networks
• R2D2: Repeatable and Reliable Detector and Descriptor
• Learning Two-View Correspondences and Geometry Using Order-Aware Network

Cited by

• FPC-Net: Revisiting SuperPoint with Descriptor-Free Keypoint Detection via Feature Pyramids and Consistency-Based Implicit Matching
• PanMatch: Unleashing the Potential of Large Vision Models for Unified Matching Models
• Diff²I2P: Differentiable Image-to-Point Cloud Registration with Diffusion Prior
• Deep Learning Reforms Image Matching: A Survey and Outlook
• Learnable Motion Coherence for Correspondence Pruning
• Focus What Matters: Matchability-Based Reweighting for Local Feature Matching
• Semantic Correspondence: Unified Benchmarking and a Strong Baseline
• SGFormer: Structure-Guided Transformer for Robust Local Feature Matching

Related

• Neighbourhood Consensus Networks
• Dual-Resolution Correspondence Networks
• Correspondence Networks with Adaptive Neighbourhood Consensus
• \mathbbXResolution Correspondence Networks
• S2DNet: Learning Accurate Correspondences for Sparse-to-Dense Feature Matching