2015/05/13 by Yi, Xinyang, Wang, Zhaoran, Caramanis, Constantine +1
#FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (stat.ML)
paper · doi:10.48550/arxiv.1505.03257
Linear regression studies the problem of estimating a model parameter β^* ∈ ℝp, from n observations \(yi,xi)\i=1n from linear model yi = ⟨ xi,β^* ⟩ + εi. We consider a significant generalization in which the relationship between ⟨ xi,β^* ⟩ and yi is noisy, quantized to a single bit, potentially nonlinear, noninvertible, as well as unknown. This model is known as the single-index model in statistics, and, among other things, it represents a significant generalization of one-bit compressed sensing. We propose a novel spectral-based estimation procedure and show that we can recover β^* in settings (i.e., classes of link function f) where previous algorithms fail. In general, our algorithm requires only very mild restrictions on the (unknown) functional relationship between yi and ⟨ xi,β^* ⟩. We also consider the high dimensional setting where β^* is sparse ,and introduce a two-stage nonconvex framework that addresses estimation challenges in high dimensional regimes where p ≫ n. For a broad class of link functions between ⟨ xi,β^* ⟩ and yi, we establish minimax lower bounds that demonstrate the optimality of our estimators in both the classical and high dimensional regimes.