Mean and variance estimation in high-dimensional heteroscedastic models with non-convex penalties

2014/10/29 by Sharpnack, James, Kolar, Mladen
#62G08 #FOS: Computer and information sciences #FOS: Mathematics #Methodology (stat.ME) #Statistics Theory (math.ST)

paper · doi:10.48550/arxiv.1410.7874

Abstract

Despite its prevalence in statistical datasets, heteroscedasticity (non-constant sample variances) has been largely ignored in the high-dimensional statistics literature. Recently, studies have shown that the Lasso can accommodate heteroscedastic errors, with minor algorithmic modifications (Belloni et al., 2012; Gautier and Tsybakov, 2013). In this work, we study heteroscedastic regression with linear mean model and log-linear variances model with sparse high-dimensional parameters. In this work, we propose estimating variances in a post-Lasso fashion, which is followed by weighted-least squares mean estimation. These steps employ non-convex penalties as in Fan and Li (2001), which allows us to prove oracle properties for both post-Lasso variance and mean parameter estimates. We reinforce our theoretical findings with experiments.

Related

• Variance function estimation in high-dimensions
• Support estimation in high-dimensional heteroscedastic mean regression
• Learning Heteroscedastic Models by Convex Programming under Group Sparsity
• High-dimensional regression with unknown variance
• Greedy Variance Estimation for the LASSO