Mamba-FETrack V2: Revisiting State Space Model for Frame-Event based Visual Object Tracking
2025/06/30 by Shiao Wang, Ju Huang, Wang, Shiao +13 · 1 citation
Computer Science · Engineering · #Advanced Memory and Neural Computing #Advanced Neural Network Applications #Age of Information Optimization #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG)
paper · pdf · doi:10.48550/arxiv.2506.23783
openalex publication_date 2025/06/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
Combining traditional RGB cameras with bio-inspired event cameras for robust object tracking has garnered increasing attention in recent years. However, most existing multimodal tracking algorithms depend heavily on high-complexity Vision Transformer architectures for feature extraction and fusion across modalities. This not only leads to substantial computational overhead but also limits the effectiveness of cross-modal interactions. In this paper, we propose an efficient RGB-Event object tracking framework based on the linear-complexity Vision Mamba network, termed Mamba-FETrack V2. Specifically, we first design a lightweight Prompt Generator that utilizes embedded features from each modality, together with a shared prompt pool, to dynamically generate modality-specific learnable prompt vectors. These prompts, along with the modality-specific embedded features, are then fed into a Vision Mamba-based FEMamba backbone, which facilitates prompt-guided feature extraction, cross-modal interaction, and fusion in a unified manner. Finally, the fused representations are passed to the tracking head for accurate target localization. Extensive experimental evaluations on multiple RGB-Event tracking benchmarks, including short-term COESOT dataset and long-term datasets, i.e., FE108 and FELT V2, demonstrate the superior performance and efficiency of the proposed tracking framework. The source code and pre-trained models will be released on https://github.com/Event-AHU/MambaFETrack
Citations
- Two-stream Beats One-stream: Asymmetric Siamese Network for Efficient Visual Tracking
- Exploiting Multimodal Spatial-temporal Patterns for Video Object Tracking
- MambaIRv2: Attentive State Space Restoration
- MambaVision: A Hybrid Mamba-Transformer Vision Backbone
- Coupled Mamba: Enhanced Multi-modal Fusion with Coupled State Space Model
- Awesome Multi-modal Object Tracking
- TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Tracking
- Mamba-FETrack: Frame-Event Tracking via State Space Model
- A Survey on Visual Mamba
- State Space Model for New-Generation Network Alternative to Transformers: A Survey
- FusionMamba: Dynamic Feature Enhancement for Multimodal Image Fusion with Mamba
- SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Tracking
- OneTracker: Unifying Visual Object Tracking with Foundation Models and Efficient Tuning
- VMamba: Visual State Space Model
- Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model
- Explicit Visual Prompts for Visual Object Tracking
- ODTrack: Online Dense Temporal Token Learning for Visual Tracking
- Temporal Adaptive RGBT Tracking with Modality Prompt
- Mamba: Linear-Time Sequence Modeling with Selective State Spaces
- Single-Model and Any-Modality for Video Object Tracking
- HIPTrack: Visual Tracking with Historical Prompts
- Robust Object Modeling for Visual Tracking
- Cross-modal Orthogonal High-rank Augmentation for RGB-Event Transformer-trackers
- Frame-Event Alignment and Fusion Network for High Frame Rate Tracking
- Generalized Relation Modeling for Transformer Tracking
- Visual Prompt Multi-Modal Tracking
- Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric
- AiATrack: Attention in Attention for Transformer Visual Tracking
- Visual Prompt Tuning
- Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework
- MixFormer: End-to-End Tracking with Iterative Mixed Attention
- Transforming Model Prediction for Tracking
- Backbone is All Your Need: A Simplified Architecture for Visual Object Tracking
- Efficiently Modeling Long Sequences with Structured State Spaces
- Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers
- Object Tracking by Jointly Exploiting Frame and Event Domain
- Learning to Prompt for Vision-Language Models
- VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
- Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
- Learning Spatio-Temporal Transformer for Visual Tracking
- Learning Target Candidate Association to Keep Track of What Not to Track
- Transformer Tracking
- Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking
- Learning Transferable Visual Models From Natural Language Supervision
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
- HiPPO: Recurrent Memory with Optimal Polynomial Projections
- Probabilistic Regression for Visual Tracking
- Know Your Surroundings: Exploiting Scene Information for Object Tracking
- PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Siam R-CNN: Visual Tracking by Re-Detection
- SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines
- Learning Discriminative Model Prediction for Tracking
- ATOM: Accurate Tracking by Overlap Maximization
- REBAR: Low-variance, unbiased gradient estimates for discrete latent\n variable models
- Gradient-based learning applied to document recognition
Cited by
Related