VirDA: Reusing Backbone for Unsupervised Domain Adaptation with Visual Reprogramming

2025/10/02 by Duy Nguyen, Nguyen, Duy, Dat Ba Nguyen +1
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Human Pose and Action Recognition #Multimodal Machine Learning Applications

paper · pdf · doi:10.48550/arxiv.2510.01660

openalex publication_date 2025/10/02 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Existing UDA pipelines fine-tune already well-trained backbone parameters for every new source-and-target pair, resulting in the number of training parameters and storage memory growing linearly with each new pair, and also preventing the reuse of these well-trained backbone parameters. Inspired by recent implications that existing backbones have textural biases, we propose making use of domain-specific textural bias for domain adaptation via visual reprogramming, namely VirDA. Instead of fine-tuning the full backbone, VirDA prepends a domain-specific visual reprogramming layer to the backbone. This layer produces visual prompts that act as an added textural bias to the input image, adapting its "style" to a target domain. To optimize these visual reprogramming layers, we use multiple objective functions that optimize the intra- and inter-domain distribution differences when domain-adapting visual prompts are applied. This process does not require modifying the backbone parameters, allowing the same backbone to be reused across different domains. We evaluate VirDA on Office-31 and obtain 92.8% mean accuracy with only 1.5M trainable parameters. VirDA surpasses PDA, the state-of-the-art parameter-efficient UDA baseline, by +1.6% accuracy while using just 46% of its parameters. Compared with full-backbone fine-tuning, VirDA outperforms CDTrans and FixBi by +0.2% and +1.4%, respectively, while requiring only 1.7% and 2.8% of their trainable parameters. Relative to the strongest current methods (PMTrans and TVT), VirDA uses ~1.7% of their parameters and trades off only 2.2% and 1.1% accuracy, respectively.

Citations

• Attribute-based Visual Reprogramming for Vision-Language Models
• Bayesian-guided Label Mapping for Visual Reprogramming
• Efficient unsupervised domain adaptation via self-supervised vision transformer and synergistic cross-domain alignment
• Sample-specific Masks for Visual Reprogramming-based Prompting
• Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation
• Prompt-based Distribution Alignment for Unsupervised Domain Adaptation
• Patch-Mix Transformer for Unsupervised Domain Adaptation: A Game Perspective
• Visual Prompt Tuning for Test-time Domain Adaptation
• MaPLe: Multi-modal Prompt Learning
• Vision GNN: An Image is Worth Graph of Nodes
• Safe Self-Refinement for Transformer-based Domain Adaptation
• Domain Adaptation via Prompt Learning
• Cross-Domain Adaptive Teacher for Object Detection
• CDTrans: Cross-domain Transformer for Unsupervised Domain Adaptation
• TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation
• A Bit More Bayesian: Domain-Invariant Learning with Uncertainty
• Prototypical Cross-domain Self-supervised Learning for Few-shot Unsupervised Domain Adaptation
• Coordinate Attention for Efficient Mobile Network Design
• Coordinate Attention for Efficient Mobile Network Design
• FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation
• An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
• Domain Conditioned Adaptation Network
• Gradually Vanishing Bridge for Adversarial Domain Adaptation
• Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations
• Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation
• Bayesian Uncertainty Matching for Unsupervised Domain Adaptation
• Bridging Theory and Algorithm for Domain Adaptation
• Progressive Feature Alignment for Unsupervised Domain Adaptation
• ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
• Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation
• Maximum Classifier Discrepancy for Unsupervised Domain Adaptation
• CyCADA: Cycle-Consistent Adversarial Domain Adaptation
• Deep Hashing Network for Unsupervised Domain Adaptation
• Generate To Adapt: Aligning Domains using Generative Adversarial Networks
• Adversarial Discriminative Domain Adaptation
• Correlation Alignment for Unsupervised Domain Adaptation
• Deep Transfer Learning with Joint Adaptation Networks
• Deep Residual Learning for Image Recognition
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Learning Transferable Features with Deep Adaptation Networks
• Unsupervised Domain Adaptation by Backpropagation
• ImageNet classification with deep convolutional neural networks
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
• Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Related

• Efficient unsupervised domain adaptation via self-supervised vision transformer and synergistic cross-domain alignment
• Improving Source-Free Target Adaptation with Vision Transformers Leveraging Domain Representation Images
• Domain2Vec: Domain Embedding for Unsupervised Domain Adaptation
• SKADA-Bench: Benchmarking Unsupervised Domain Adaptation Methods with Realistic Validation On Diverse Modalities
• CoDA: Instructive Chain-of-Domain Adaptation with Severity-Aware Visual Prompt Tuning