Similarity-Aware Selective State-Space Modeling for Semantic Correspondence

2025/09/29 by Kim, Seungwook, Cho, Minsu
#Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences

paper · doi:10.48550/arxiv.2509.24318

Abstract

Establishing semantic correspondences between images is a fundamental yet challenging task in computer vision. Traditional feature-metric methods enhance visual features but may miss complex inter-correlation relationships, while recent correlation-metric approaches are hindered by high computational costs due to processing 4D correlation maps. We introduce MambaMatcher, a novel method that overcomes these limitations by efficiently modeling high-dimensional correlations using selective state-space models (SSMs). By implementing a similarity-aware selective scan mechanism adapted from Mamba's linear-complexity algorithm, MambaMatcher refines the 4D correlation map effectively without compromising feature map resolution or receptive field. Experiments on standard semantic correspondence benchmarks demonstrate that MambaMatcher achieves state-of-the-art performance.

Citations

• Learning SO(3)-Invariant Semantic Correspondence via Local Shape Transform
• CorrespondentDream: Enhancing 3D Fidelity of Text-to-3D using Cross-View Correspondences
• PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition
• Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
• VideoMamba: State Space Model for Efficient Video Understanding
• Point Mamba: A Novel Point Cloud Backbone Based on State Space Model with Octree-Based Ordering Strategy
• MedMamba: Vision Mamba for Medical Image Classification
• VM-UNet: Vision Mamba UNet for Medical Image Segmentation
• VMamba: Visual State Space Model
• Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
• Point Transformer V3: Simpler, Faster, Stronger
• StableVITON: Learning Semantic Correspondence with Latent Diffusion Model for Virtual Try-On
• Mamba: Linear-Time Sequence Modeling with Selective State Spaces
• Efficient Semantic Matching with Hypercolumn Correlation
• SD4Match: Learning to Prompt Stable Diffusion Model for Semantic Matching
• Emergent Correspondence from Image Diffusion
• A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence
• Unsupervised Semantic Correspondence Using Stable Diffusion
• Diffusion Hyperfeatures: Searching Through Time and Space for Semantic Correspondence
• DINOv2: Learning Robust Visual Features without Supervision
• Neural Matching Fields: Implicit Representation of Matching Fields for Visual Correspondence
• Learning Semantic Correspondence with Sparse Annotations
• Simplified State Space Layers for Sequence Modeling
• TransforMatcher: Match-to-Match Attention for Semantic Correspondence
• CATs++: Boosting Cost Aggregation with Convolutions and Transformers
• High-Resolution Image Synthesis with Latent Diffusion Models
• High-Resolution Image Synthesis with Latent Diffusion Models
• GAN-Supervised Dense Visual Alignment
• Efficiently Modeling Long Sequences with Structured State Spaces
• Convolutional Hough Matching Networks for Robust and Efficient Visual Correspondence
• Multi-scale Matching Networks for Semantic Correspondence
• Convolutional Hough Matching Networks
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• HiPPO: Recurrent Memory with Optimal Polynomial Projections
• Learning to Compose Hypercolumns for Visual Correspondence
• Correspondence Networks with Adaptive Neighbourhood Consensus
• SPair-71k: A Large-scale Benchmark for Semantic Correspondence
• SFNet: Learning Object-aware Semantic Correspondence
• Neighbourhood Consensus Networks
• Attentive Semantic Alignment with Offset-Aware Correlation Kernels
• Parallelizing Linear Recurrent Neural Nets Over Sequence Length
• Proposal Flow: Semantic Correspondences from Object Proposals
• Deep Residual Learning for Image Recognition
• Dense Semantic Correspondence where Every Pixel is a Classifier
• Unsupervised Object Discovery and Localization in the Wild: Part-based Matching with Bottom-up Region Proposals
• Adam: A Method for Stochastic Optimization
• SIFT Flow: Dense Correspondence across Scenes and Its Applications
• Distinctive Image Features from Scale-Invariant Keypoints
• Fastformer: Additive Attention Can Be All You Need

Related

• Mamba-Adaptor: State Space Model Adaptor for Visual Recognition
• Dense Semantic Correspondence where Every Pixel is a Classifier
• PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition
• MambaGlue: Fast and Robust Local Feature Matching With Mamba
• Mamba2D: A Natively Multi-Dimensional State-Space Model for Vision Tasks