2018/07/06 by Amichai Painsky, Meir Feder, Painsky, Amichai +1
Biochemistry, Genetics and Molecular Biology · Computer Science · #Algorithms and Data Compression #FOS: Computer and information sciences #Fractal and DNA sequence analysis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications
paper · pdf · doi:10.48550/arxiv.1807.02287
openalex publication_date 2018/07/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Estimating a large alphabet probability distribution from a limited number of samples is a fundamental problem in machine learning and statistics. A variety of estimation schemes have been proposed over the years, mostly inspired by the early work of Laplace and the seminal contribution of Good and Turing. One of the basic assumptions shared by most commonly-used estimators is the unique correspondence between the symbol's sample frequency and its estimated probability. In this work we tackle this paradigmatic assumption; we claim that symbols with "similar" frequencies shall be assigned the same estimated probability value. This way we regulate the number of parameters and improve generalization. In this preliminary report we show that by applying an ensemble of such regulated estimators, we introduce a dramatic enhancement in the estimation accuracy (typically up to 50%), compared to currently known methods. An implementation of our suggested method is publicly available at the first author's web-page.