Distributionally Robust Multimodal Machine Learning

2025/11/07 by Yang, Peilin, Ma, Yu
Computer Science · Decision Sciences · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Risk and Portfolio Optimization #Stochastic Gradient Optimization Techniques

paper · doi:10.48550/arxiv.2511.05716

openalex publication_date 2025/11/07 · openalex created_date 2025/11/12 · openalex updated_date 2026/07/28

Abstract

We consider the problem of distributionally robust multimodal machine learning. Existing approaches often rely on merging modalities on the feature level (early fusion) or heuristic uncertainty modeling, which downplays modality-aware effects and provide limited insights. We propose a novel distributionally robust optimization (DRO) framework that aims to study both the theoretical and practical insights of multimodal machine learning. We first justify this setup and show the significance of this problem through complexity analysis. We then establish both generalization upper bounds and minimax lower bounds which provide performance guarantees. These results are further extended in settings where we consider encoder-specific error propogations. Empirically, we demonstrate that our approach improves robustness in both simulation settings and real-world datasets. Together, these findings provide a principled foundation for employing multimodal machine learning models in high-stakes applications where uncertainty is unavoidable.

Citations

• DRO: A Python Library for Distributionally Robust Optimization in Machine Learning
• Creating a Cooperative AI Policymaking Platform through Open Source Collaboration
• Randomized Exploration for Reinforcement Learning with Multinomial Logistic Function Approximation
• Beyond Unimodal: Generalising Neural Processes for Multimodal Uncertainty Estimation
• Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework
• Tractable Uncertainty-Aware Meta-Learning
• Distributionally Robust Weighted k-Nearest Neighbors
• Regularization via Mass Transportation
• Distributionally Robust Logistic Regression
• The information bottleneck method

Related

• Distribution-Dependent Rates for Multi-Distribution Learning
• Globalized distributionally robust optimization with multi core sets
• Learning Models with Uniform Performance via Distributionally Robust\n Optimization
• Modeling the Second Player in Distributionally Robust Optimization
• Learning Distributionally Robust Models at Scale via Composite Optimization