When is Importance Weighting Correction Needed for Covariate Shift Adaptation?

2023/03/07 by Davit Gogolashvili, Matteo Zecchin, Gogolashvili, Davit +7 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · Mathematics · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Genetic and phenotypic traits in livestock #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistical Methods and Inference

paper · pdf · doi:10.48550/arxiv.2303.04020

openalex publication_date 2023/03/07 · openalex created_date 2023/03/10 · openalex updated_date 2026/07/30

Abstract

This paper investigates when the importance weighting (IW) correction is needed to address covariate shift, a common situation in supervised learning where the input distributions of training and test data differ. Classic results show that the IW correction is needed when the model is parametric and misspecified. In contrast, recent results indicate that the IW correction may not be necessary when the model is nonparametric and well-specified. We examine the missing case in the literature where the model is nonparametric and misspecified, and show that the IW correction is needed for obtaining the best approximation of the true unknown function for the test distribution. We do this by analyzing IW-corrected kernel ridge regression, covering a variety of settings, including parametric and nonparametric models, well-specified and misspecified settings, and arbitrary weighting functions.

Citations

• Domain Adaptation: Learning Bounds and Algorithms
• Information-Based Objective Functions for Active Data Selection
• Sample Selection Bias as a Specification Error
• Generalization Properties of Learning with Random Features
• What is the Effect of Importance Weighting in Deep Learning?
• Understanding Why Generalized Reweighting Does Not Improve Over ERM
• Is Importance Weighting Incompatible with Interpolating Classifiers?
• Pseudo-Labeling for Kernel Ridge Regression under Covariate Shift

Cited by

• Spectral Algorithms in Misspecified Regression: Convergence under Covariate Shift

Related

• Importance Weighting Correction of Regularized Least-Squares for Covariate and Target Shifts
• Robust Importance Weighting for Covariate Shift
• Robust Covariate Shift Prediction with General Losses and Feature Views
• A One-step Approach to Covariate Shift Adaptation
• Extreme Dimension Reduction for Handling Covariate Shift