2004/12/31 by Marc Coram, Steven P. Lalley
Computer Science · Mathematics · #Bayesian Methods and Mixture Models #Statistical Methods and Inference #Statistical Methods in Clinical Trials #math.ST #msc:62A15 #msc:62E20 #stat.TH
paper · pdf · doi:10.1214/009053606000000236
published as Annals of Statistics 2006, Vol. 34, No. 3, 1233-1269 · Published at http://dx.doi.org/10.1214/009053606000000236 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)
openalex publication_date 2006/06/01 · arxiv created 2006/07/31 · arxiv updated 2009/12/01 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28
When do nonparametric Bayesian procedures “overfit”? To shed light on this question, we consider a binary regression problem in detail and establish frequentist consistency for a certain class of Bayes procedures based on hierarchical priors, called uniform mixture priors. These are defined as follows: let ν be any probability distribution on the nonnegative integers. To sample a function f from the prior πν, first sample m from ν and then sample f uniformly from the set of step functions from [0,1] into [0,1] that have exactly m jumps (i.e., sample all m jump locations and m+1 function values independently and uniformly). The main result states that if a data-stream is generated according to any fixed, measurable binary-regression function f0≢1/2, then frequentist consistency obtains: that is, for any ν with infinite support, the posterior of πν concentrates on any L1 neighborhood of f0. Solution of an associated large-deviations problem is central to the consistency proof.