Asymptotics for regression models under loss of identifiability

2013/09/16 by Joseph Rynkiewicz, Rynkiewicz, Joseph
Computer Science · Engineering · Mathematics · #Control Systems and Identification #FOS: Mathematics #Neural Networks and Applications #Statistical Methods and Inference #Statistics Theory (math.ST) #math.ST #stat.TH

paper · pdf · doi:10.48550/arxiv.1309.3912

arxiv created 2013/09/16 · openalex publication_date 2013/09/16 · arxiv updated 2013/09/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

This paper discusses the asymptotic behavior of regression models under general conditions. First, we give a general inequality for the difference of the sum of square errors (SSE) of the estimated regression model and the SSE of the theoretical best regression function in our model. A set of generalized derivative functions is a key tool in deriving such inequality. Under suitable Donsker condition for this set, we give the asymptotic distribution for the difference of SSE. We show how to get this Donsker property for parametric models even if the parameters characterizing the best regression function are not unique. This result is applied to neural networks regression models with redundant hidden units when loss of identifiability occurs.

Related

• Non-identifiability distinguishes Neural Networks among Parametric Models
• Errors-in-variables models: a generalized functions approach
• Generalization Error of Generalized Linear Models in High Dimensions
• On the Universality of the Double Descent Peak in Ridgeless Regression
• The generalization error of random features regression: Precise asymptotics and double descent curve