Transfer Learning for Nonparametric Regression: Non-asymptotic Minimax Analysis and Adaptive Procedure

2024/01/22 by Cai, T. Tony, Pu, Hongming · 7 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · doi:10.48550/arxiv.2401.12272

Abstract

Transfer learning for nonparametric regression is considered. We first study the non-asymptotic minimax risk for this problem and develop a novel estimator called the confidence thresholding estimator, which is shown to achieve the minimax optimal risk up to a logarithmic factor. Our results demonstrate two unique phenomena in transfer learning: auto-smoothing and super-acceleration, which differentiate it from nonparametric regression in a traditional setting. We then propose a data-driven algorithm that adaptively achieves the minimax risk up to a logarithmic factor across a wide range of parameter spaces. Simulation studies are conducted to evaluate the numerical performance of the adaptive transfer learning algorithm, and a real-world example is provided to demonstrate the benefits of the proposed method.

Cited by

• Cross-Semantic Transfer Learning for High-Dimensional Linear Regression
• Transfer Learning for Benign Overfitting in High-Dimensional Linear Regression
• Transfer Learning with Distance Covariance for Random Forest: Error Bounds and an EHR Application
• Limitations of refinement methods for weak to strong generalization
• Sufficiency-principled Transfer Learning via Model Averaging
• Late Fusion Multi-task Learning for Semiparametric Inference with Nuisance Parameters
• Transfer Learning for Matrix Completion

Related

• Model-Robust and Adaptive-Optimal Transfer Learning for Tackling Concept Shifts in Nonparametric Regression
• Transfer Learning for Nonparametric Classification: Minimax Rate and Adaptive Classifier
• Minimax optimal transfer learning for high-dimensional additive regression
• Transfer Learning for Kernel-based Regression
• Augmented Transfer Regression Learning with Semi-non-parametric Nuisance Models