Non-Data-Splitting Estimator Selection for Regression in Exponential Families

2022/12/25 by Juntong Chen, Chen, Juntong
Mathematics · Biochemistry, Genetics and Molecular Biology · #Statistical Methods and Inference #Advanced Causal Inference Techniques #Gene expression and cancer classification

paper · pdf · doi:10.48550/arxiv.2212.12954

Abstract

We observe n independent pairs of random variables (W_i, Y_i), where the conditional distribution of Y_i given W_i=w_i follows a one-parameter exponential family with parameter \bsg^*(w_i)∈\R. Our goal is to estimate the regression function \bsg^*. We start with an arbitrary collection of piecewise constant candidate estimators based on our observations and, using the same data, select an estimator from this collection. Our approach is agnostic to the dependencies of the candidate estimators on the data, differing from methods like data splitting, cross-validation, and hold-out. To demonstrate its theoretical performance, we provide a non-asymptotic risk bound for the selected estimator. We then explain how to apply the procedure to changepoint detection in exponential families. The practical performance of the proposed approach is illustrated through a comparative simulation study under different scenarios and real datasets.

Related

• Robust estimation of a regression function in exponential families
• Estimating a regression function in exponential families by model\n selection
• Model selection for the segmentation of multiparameter exponential family distributions
• Novel closed-form point estimators for a weighted exponential family derived from likelihood equations
• Estimating Densities with Non-Parametric Exponential Families