2016/01/31 by Jonathan M. Friedman, Friedman, Jonathan Malcolm
Computer Science · Decision Sciences · Mathematics · #Advanced Statistical Process Monitoring #Bayesian Modeling and Causal Inference #Computation (stat.CO) #FOS: Computer and information sciences #Methodology (stat.ME) #Statistical Methods and Bayesian Inference
paper · pdf · doi:10.48550/arxiv.1602.00207
openalex publication_date 2016/01/31 · openalex created_date 2016/06/24 · openalex updated_date 2026/07/28
Misestimates of σPo, the uncertainty in Po from a 2-state Bayes equation used for binary classification, apparently arose from σpi, the uncertainty in underlying pdfs estimated from experimental b-bin histograms. To address this, several Bayesian estimator pairs (pi, σpi) were compared for agreement between nominal confidence level (ξ) and calculated coverage values (C). Large ξ-to-C inconsistency for large b and pi ≫ (1)/(b) arises for all multinomial estimators since priors downweight low likelihood, high pi values. To improve ξ-to-C matching, (ξ-C)2 was minimized against α0 in a more general prior pdf (B[α0,(b-1)α0;x]) to obtain (pi)ξ↔ C. This improved matching for b=2, but for b>2, ξ-to-C matching by (pi)ξ↔ C required an effective value "b=2" and renormalization, and this reduced pi-to-pi matching. Better pi-to-pi matching came from the original multinomial estimators, a new discrete-domain estimator p(ni,N), or an earlier joint estimator, (pi)\bowtie that co-adjusted all estimates pi for James-Stein shrinkage to a mean vector. Best simultaneous ξ-to-C and pi-to-pi matching came by de-noising initial estimates of underlying pdfs. For b=100, N<12800, de-noised p needed ≈ 10× fewer observations to achieve pi-to-pi matching equivalent to that found for p(ni,N), (pi)\bowtie or the original multinomial pi. De-noising each different type of initial estimate yielded similarly high accuracy in Monte-Carlo tests.