Graph-Smoothed Bayesian Black-Box Shift Estimator and Its Information Geometry

2025/05/22 by Masanari Kimura, Kimura, Masanari
Computer Science · Engineering · #Advanced Graph Neural Networks #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Sparse and Compressive Sensing Techniques

paper · pdf · doi:10.48550/arxiv.2505.16251

openalex publication_date 2025/05/22 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Label shift adaptation aims to recover target class priors when the labelled source distribution P and the unlabelled target distribution Q share P(X | Y) = Q(X | Y) but P(Y) ≠ Q(Y). Classical black-box shift estimators invert an empirical confusion matrix of a frozen classifier, producing a brittle point estimate that ignores sampling noise and similarity among classes. We present Graph-Smoothed Bayesian BBSE (GS-B³SE), a fully probabilistic alternative that places Laplacian-Gaussian priors on both target log-priors and confusion-matrix columns, tying them together on a label-similarity graph. The resulting posterior is tractable with HMC or a fast block Newton-CG scheme. We prove identifiability, N^-1/2 contraction, variance bounds that shrink with the graph's algebraic connectivity, and robustness to Laplacian misspecification. We also reinterpret GS-B³SE through information geometry, showing that it generalizes existing shift estimators.

Citations

• Density Ratio Estimation via Sampling along Generalized Geodesics on Statistical Manifolds
• A Short Survey on Importance Weighting for Machine Learning
• Label Shift Adapter for Test-Time Adaptation under Covariate and Label Shifts
• A Comprehensive Survey on Test-Time Adaptation Under Distribution Shifts
• RLSbench: Domain Adaptation Under Relaxed Label Shift
• Geometry of EM and related iterative algorithms
• Mandoline: Model Evaluation under Distribution Shift
• Learning Transferable Visual Models From Natural Language Supervision
• WILDS: A Benchmark of in-the-Wild Distribution Shifts
• A Unified View of Label Shift Estimation
• Regularized Learning for Domain Adaptation under Label Shifts
• Failing Loudly: An Empirical Study of Methods for Detecting Dataset Shift
• Detecting and Correcting for Label Shift with Black Box Predictors
• A Conceptual Introduction to Hamiltonian Monte Carlo
• Deep Residual Learning for Image Recognition
• Hamiltonian Monte Carlo for Hierarchical Models
• The MNIST Database of Handwritten Digit Images for Machine Learning Research [Best of the Web]
• Natural Gradient Works Efficiently in Learning
• Adjusting the Outputs of a Classifier to New aPrioriProbabilities: A Simple Procedure

Related

• Detecting and Correcting for Label Shift with Black Box Predictors
• Nonparametric Bayesian label prediction on a large graph using truncated\n Laplacian regularization
• Maximum Likelihood with Bias-Corrected Calibration is Hard-To-Beat at Label Shift Adaptation
• A Unified View of Label Shift Estimation
• Optimal Bayesian Smoothing of Functional Observations over a Large Graph