Vector preference-based contextual bandits under distributional shifts

2025/08/21 by Shukla, Apurv, Kumar, P. R.
#FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Probability (math.PR) #Systems and Control (eess.SY) #electronic engineering #information engineering

paper · doi:10.48550/arxiv.2508.15966

Abstract

We consider contextual bandit learning under distribution shift when reward vectors are ordered according to a given preference cone. We propose an adaptive-discretization and optimistic elimination based policy that self-tunes to the underlying distribution shift. To measure the performance of this policy, we introduce the notion of preference-based regret which measures the performance of a policy in terms of distance between Pareto fronts. We study the performance of this policy by establishing upper bounds on its regret under various assumptions on the nature of distribution shift. Our regret bounds generalize known results for the existing case of no distribution shift and vectorial reward settings, and scale gracefully with problem parameters in presence of distribution shifts.

Citations

• Drug Discovery under Covariate Shift with Domain-Informed Prior Distributions over Functions
• Adaptive Algorithms for Relaxed Pareto Set Identification
• Transfer Learning for Contextual Multi-armed Bandits
• A new similarity measure for covariate shift with applications to nonparametric regression
• Learning Under Adversarial and Interventional Shifts
• Self-Tuning Bandits over Unknown Covariate-Shifts
• Bandit Algorithms
• Smooth Contextual Bandits: Bridging the Parametric and Non-differentiable Regret Regimes
• Hedging the Drift: Learning to Optimize under Non-Stationarity
• Multi-objective Contextual Bandit Problem with Similarity Information
• Satisficing in Time-Sensitive Bandit Learning
• Marginal Singularity, and the Benefits of Labels in Covariate-Shift
• Bandits and Experts in Metric Spaces
• Nonparametric Bandits with Covariates
• Contextual Bandits with Similarity Information
• Fast learning rates for plug-in classifiers under the margin condition

Related

• Contextual Bandits with Continuous Actions: Smoothing, Zooming, and Adapting
• Satisficing Regret Minimization in Bandits: Constant Rate and Light-Tailed Distribution
• Smooth Contextual Bandits: Bridging the Parametric and Non-differentiable Regret Regimes
• Meta-Learning Bandit Policies by Gradient Ascent
• Worst-case Performance of Greedy Policies in Bandits with Imperfect Context Observations