Black-Box Model Confidence Sets Using Cross-Validation with High-Dimensional Gaussian Comparison

2022/11/09 by Nicholas Kissel, Jing Lei, Kissel, Nicholas +1 · 1 citation
Decision Sciences · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Methodology (stat.ME) #Probabilistic and Robust Engineering Design #Probability and Risk Models #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.2211.04958

openalex publication_date 2022/11/09 · openalex created_date 2022/11/15 · openalex updated_date 2026/07/28

Abstract

We derive high-dimensional Gaussian comparison results for the standard V-fold cross-validated risk estimates. Our results combine a recent stability-based argument for the low-dimensional central limit theorem of cross-validation with the high-dimensional Gaussian comparison framework for sums of independent random variables. These results give new insights into the joint sampling distribution of cross-validated risks in the context of model comparison and tuning parameter selection, where the number of candidate models and tuning parameters can be larger than the fitting sample size. As a consequence, our results provide theoretical support for a recent methodological development that constructs model confidence sets using cross-validation.

Cited by

• Throwing Vines at the Wall: Structure Learning via Random Search

Related

• A Modern Theory of Cross-Validation through the Lens of Stability
• The Structure of Cross-Validation Error: Stability, Covariance, and Minimax Limits
• Cross-Validation, Risk Estimation, and Model Selection
• Concentration inequalities of the cross-validation estimate for stable predictors
• Asymptotics of Cross-Validation