2018/10/04 by Michał Dereziński, Dereziński, Michał, Manfred K. Warmuth +3 · 1 citation
Computer Science · Mathematics · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Medical Image Segmentation Techniques #Statistical Methods and Inference
paper · pdf · doi:10.48550/arxiv.1810.02453
openalex publication_date 2018/10/04 · openalex created_date 2018/10/12 · openalex updated_date 2026/07/28
Consider linear regression where the examples are generated by an unknown distribution on Rd× R. Without any assumptions on the noise, the linear least squares solution for any i.i.d. sample will typically be biased w.r.t. the least squares optimum over the entire distribution. However, we show that if an i.i.d. sample of any size k is augmented by a certain small additional sample, then the solution of the combined sample becomes unbiased. We show this when the additional sample consists of d points drawn jointly according to the input distribution that is rescaled by the squared volume spanned by the points. Furthermore, we propose algorithms to sample from this volume-rescaled distribution when the data distribution is only known through an i.i.d sample.