Rethinking Efficient Mixture-of-Experts for Remote Sensing Modality-Missing Classification

2025/11/14 by Qi Gao, John J. Qu, Gao, Qinghao +4
Computer Science · Engineering · #Advanced Neural Network Applications #Computation #Computer Vision and Pattern Recognition (cs.CV) #Contextual image classification #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generalization #Perspective (graphical) #Remote-Sensing Image Classification #Robustness (evolution) #Scalability

paper · pdf · doi:10.48550/arxiv.2511.11460

published in arXiv (Cornell University) (Cornell University)

openalex publication_date 2025/11/14 · openalex created_date 2025/11/18 · openalex updated_date 2026/08/05

Abstract

Multimodal remote sensing classification often suffers from missing modalities caused by sensor failures and environmental interference, leading to severe performance degradation. In this work, we rethink missing-modality learning from a conditional computation perspective and investigate whether Mixture-of-Experts (MoE) models can inherently adapt to diverse modality-missing scenarios. We first conduct a systematic study of representative MoE paradigms under various missing-modality settings, revealing both their potential and limitations. Building on these insights, we propose a Missing-aware Mixture-of-LoRAs (MaMOL), a parameter-efficient MoE framework that unifies multiple modality-missing cases within a single model. MaMOL introduces a dual-routing mechanism to decouple modality-invariant shared experts and modality-aware dynamic experts, enabling automatic expert activation conditioned on available modalities. Extensive experiments on multiple remote sensing benchmarks demonstrate that MaMOL significantly improves robustness and generalization under diverse missing-modality scenarios with minimal computational overhead. Transfer experiments on natural image datasets further validate its scalability and cross-domain applicability.

Citations

• Synergistic Prompting for Robust Visual Recognition with Missing Modalities
• Post-pre-training for Modality Alignment in Vision-Language Foundation Models
• Knowledge Bridger: Towards Training-free Missing Modality Completion
• COMO: Cross-Mamba Interaction and Offset-Guided Fusion for Multimodal Object Detection
• Not Just Text: Uncovering Vision Modality Typographic Threats in Image Generation Models
• Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging
• ReMix: Training Generalized Person Re-identification on a Mixture of Data
• Moirai-MoE: Empowering Time Series Foundation Models with Sparse Mixture of Experts
• Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts
• Deep Correlated Prompting for Visual Recognition with Missing Modalities
• Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate
• Missing Modality Prediction for Unpaired Multimodal Learning via Joint Embedding of Unimodal Models
• Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts
• Quantifying Uncertainty in Motion Prediction with Variational Bayesian Mixture
• Multi-Task Dense Prediction via Mixture of Low-Rank Experts
• Task-Customized Mixture of Adapters for General Image Fusion
• Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters
• FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion
• SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery
• Multimodal Industrial Anomaly Detection by Crossmodal Feature Mapping
• Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mixture of Low-rank Experts
• When MOE Meets LLMs: Parameter Efficient Fine-tuning for Multi-task Medical Applications
• Domain Generalization Using Large Pretrained Models with Mixture-of-Adapters
• Preserving Modality Structure Improves Multi-Modal Learning
• Enhancing NeRF akin to Enhancing LLMs: Generalizable NeRF Transformer with Mixture-of-View-Experts
• DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
• Multimodal Prompting with Missing Modalities for Visual Recognition
• Tag-assisted Multimodal Sentiment Analysis under Uncertain Missing Modalities
• Are Multimodal Transformers Robust to Missing Modality?
• DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale
• Scaling Vision with Sparse Mixture of Experts
• Learning Transferable Visual Models From Natural Language Supervision
• Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

Related

• Incomplete Multimodal Learning for Remote Sensing Data Fusion
• Filling the Gaps: A Multitask Hybrid Multiscale Generative Framework for Missing Modality in Remote Sensing Semantic Segmentation
• RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation
• MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Models
• CSMoE: An Efficient Remote Sensing Foundation Model with Soft Mixture-of-Experts