2022/03/13 by Juntong Chen, Chen, Juntong · 1 citation
Computer Science · Physics and Astronomy · #62G35 #Applications (stat.AP) #FOS: Computer and information sciences #FOS: Mathematics #Model Reduction and Neural Networks #Neural Networks and Applications #Primary 62G05 #Secondary 62J12 #Statistics Theory (math.ST) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2203.06656
openalex publication_date 2022/03/13 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Let X1=(W1,Y1),\…,Xn=(Wn,Yn) be n pairs of\nindependent random variables. We assume that, for each i\∈ 1,\…,n ,\nthe conditional distribution of Yi given Wi belongs to a\none-parameter exponential family with parameter\n boldsymbol\γ\⋆(Wi)\∈\ℝ, or at least, is close\nenough to a distribution of this form. The objective of the present paper is to\nestimate these conditional distributions on the basis of the observation\n boldsymbolX=(X1,\…,Xn) and to do so, we propose a model\nselection procedure together with a non-asymptotic risk bound for the resulted\nestimator with respect to a Hellinger-type distance. When\n boldsymbol\γ\⋆ does exist, the procedure allows to obtain an\nestimator widehat boldsymbol\γ of boldsymbol\γ\⋆\nadapted to a wide range of the anisotropic Besov spaces. When\n boldsymbol\γ\⋆ has a general additive or multiple index\nstructure, we construct suitable models and show the resulted estimators by our\nprocedure based on such models can circumvent the curse of dimensionality.\nMoreover, we consider model selection problems for ReLU neural networks and\nprovide an example where estimation based on neural networks enjoys a much\nfaster converge rate than the classical models. Finally, we apply this\nprocedure to solve variable selection problem in exponential families. The\nproofs in the paper rely on bounding the VC dimensions of several collections\nof functions, which can be of independent interest.\n