Post-Transfer Learning Statistical Inference in High-Dimensional Regression

2025/04/25 by Nguyen Vu Khai Tam, Tam, Nguyen Vu Khai, Cao Huyen My +3
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Face and Expression Recognition #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM

paper · pdf · doi:10.48550/arxiv.2504.18212

openalex publication_date 2025/04/25 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Transfer learning (TL) for high-dimensional regression (HDR) is an important problem in machine learning, particularly when dealing with limited sample size in the target task. However, there currently lacks a method to quantify the statistical significance of the relationship between features and the response in TL-HDR settings. In this paper, we introduce a novel statistical inference framework for assessing the reliability of feature selection in TL-HDR, called PTL-SI (Post-TL Statistical Inference). The core contribution of PTL-SI is its ability to provide valid p-values to features selected in TL-HDR, thereby rigorously controlling the false positive rate (FPR) at desired significance level α (e.g., 0.05). Furthermore, we enhance statistical power by incorporating a strategic divide-and-conquer approach into our framework. We demonstrate the validity and effectiveness of the proposed PTL-SI through extensive experiments on both synthetic and real-world high-dimensional datasets, confirming its theoretical properties and utility in testing the reliability of feature selection in TL scenarios.

Citations

• Statistical Inference for Feature Selection after Optimal Transport-based Domain Adaptation
• TransFusion: Covariate-Shift Robust Transfer Learning for High-Dimensional Regression
• CAD-DA: Controllable Anomaly Detection after Domain Adaptation by Statistical Inference
• Transfer Learning under High-dimensional Generalized Linear Models
• Transfer Learning for High-dimensional Linear Regression: Prediction, Estimation, and Minimax Optimality
• More powerful post-selection inference, with application to the Lasso
• Selective Inference for Group-Sparse Linear Models
• Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning
• Optimal Inference After Model Selection
• A significance test for forward stepwise model selection
• Exact Post-Selection Inference for Sequential Regression Procedures
• Exact post-selection inference, with application to the lasso
• A Study of Error Variance Estimation in Lasso Regression
• Nearly unbiased variable selection under minimax concave penalty

Related

• Heterogeneous transfer learning for high-dimensional regression with feature mismatch
• Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression
• Profiled Transfer Learning for High Dimensional Linear Model
• Features are fate: a theory of transfer learning in high-dimensional regression
• AdaTrans: Feature-wise and Sample-wise Adaptive Transfer Learning for High-dimensional Regression