2024/10/30 by Anders Aamand, Aamand, Anders, Alexandr Andoni +11 · 1 citation
Computer Science · Mathematics · #Bayesian Methods and Mixture Models #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistical Methods and Inference
paper · pdf · doi:10.48550/arxiv.2410.23087
openalex publication_date 2024/10/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We study the density estimation problem defined as follows: given k distributions p1, …, pk over a discrete domain [n], as well as a collection of samples chosen from a ``query'' distribution q over [n], output pi that is ``close'' to q. Recently~\citeaamand2023data gave the first and only known result that achieves sublinear bounds in \em both the sampling complexity and the query time while preserving polynomial data structure space. However, their improvement over linear samples and time is only by subpolynomial factors. Our main result is a lower bound showing that, for a broad class of data structures, their bounds cannot be significantly improved. In particular, if an algorithm uses O(n/logc k) samples for some constant c>0 and polynomial space, then the query time of the data structure must be at least k1-O(1)/log log k, i.e., close to linear in the number of distributions k. This is a novel statistical-computational trade-off for density estimation, demonstrating that any data structure must use close to a linear number of samples or take close to linear query time. The lower bound holds even in the realizable case where q=pi for some i, and when the distributions are flat (specifically, all distributions are uniform over half of the domain [n]). We also give a simple data structure for our lower bound instance with asymptotically matching upper bounds. Experiments show that the data structure is quite efficient in practice.