2013/05/31 by Jason D. Lee, Yuekai Sun, Lee, Jason D. +3
Computer Science · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Methodology (stat.ME) #Optimization and Control (math.OC) #Statistics Theory (math.ST) #cs.LG #math.OC #math.ST #stat.ME #stat.ML #stat.TH
paper · pdf · doi:10.48550/arxiv.1305.7477
arxiv created 2014/10/11 · arxiv updated 2014/10/14
Regularized M-estimators are used in diverse areas of science and engineering to fit high-dimensional models with some low-dimensional structure. Usually the low-dimensional structure is encoded by the presence of the (unknown) parameters in some low-dimensional model subspace. In such settings, it is desirable for estimates of the model parameters to be model selection consistent: the estimates also fall in the model subspace. We develop a general framework for establishing consistency and model selection consistency of regularized M-estimators and show how it applies to some special cases of interest in statistical learning. Our analysis identifies two key properties of regularized M-estimators, referred to as geometric decomposability and irrepresentability, that ensure the estimators are consistent and model selection consistent.