2020/02/28 by Benjamin G. Stokell, Rajen D. Shah, Stokell, Benjamin G. +3 · 1 citation
Computer Science · Mathematics · #62J07 #Advanced Statistical Methods and Models #Bayesian Modeling and Causal Inference #Computation (stat.CO) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (stat.ML) #Methodology (stat.ME) #Statistical Methods and Inference #Statistics Theory (math.ST)
paper · pdf · doi:10.48550/arxiv.2002.12606
openalex publication_date 2020/02/28 · openalex created_date 2022/07/26 · openalex updated_date 2026/07/28
We propose a method for estimation in high-dimensional linear models with\nnominal categorical data. Our estimator, called SCOPE, fuses levels together by\nmaking their corresponding coefficients exactly equal. This is achieved using\nthe minimax concave penalty on differences between the order statistics of the\ncoefficients for a categorical variable, thereby clustering the coefficients.\nWe provide an algorithm for exact and efficient computation of the global\nminimum of the resulting nonconvex objective in the case with a single variable\nwith potentially many levels, and use this within a block coordinate descent\nprocedure in the multivariate case. We show that an oracle least squares\nsolution that exploits the unknown level fusions is a limit point of the\ncoordinate descent with high probability, provided the true levels have a\ncertain minimum separation; these conditions are known to be minimal in the\nunivariate case. We demonstrate the favourable performance of SCOPE across a\nrange of real and simulated datasets. An R package CatReg implementing SCOPE\nfor linear models and also a version for logistic regression is available on\nCRAN.\n