Complexity Regularization and Local Metric Entropy

2016/09/09 by Fabian Latorre, Latorre, Fabián
Decision Sciences · Mathematics · #FOS: Mathematics #Mathematical Approximation and Integration #Probabilistic and Robust Engineering Design #Statistical Methods and Inference #Statistics Theory (math.ST)

paper · pdf · doi:10.48550/arxiv.1609.02855

openalex publication_date 2016/09/09 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In the context of Structural Risk Minimization, one is presented a sequence of classes \G_j\ from which, given a random sample (X_i,Y_i) one wants to choose a strongly consistent estimator. For certain types of classes of functions, we present a criterion to choose an estimator, based on the minimization of the sum of empirical error and a complexity penalty r(n,j) over each class \G_j. We present also several other results together with important results found on current literature on the subject, in an attempt to present and unify the theory in the context of regression estimation. In particular we present a generalization of the consistency of Structural Risk Minimization in the context of regression estimation which are all new results found on Chapter 5.

Related

• Empirical Risk Minimization with Relative Entropy Regularization: Optimality and Sensitivity Analysis
• Set structured global empirical risk minimizers are rate optimal in general dimensions
• Regularization and Optimal Multiclass Learning
• Asymmetry of the Relative Entropy in the Regularization of Empirical Risk Minimization
• A new analytical approach to consistency and overfitting in regularized\n empirical risk minimization