Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses

2005/01/31 by Nicolai Meinshausen, John Rice · 6 citations
Mathematics · #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistical Methods in Clinical Trials #math.ST #msc:62H15 #msc:62J15 #msc:62P35 #stat.TH

paper · pdf · doi:10.1214/009053605000000741

published as Annals of Statistics 2006, Vol. 34, No. 1, 373-393 · Published at http://dx.doi.org/10.1214/009053605000000741 in the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

openalex publication_date 2006/02/01 · arxiv created 2006/05/18 · arxiv updated 2009/12/01 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

We consider the problem of estimating the number of false null hypotheses among a very large number of independently tested hypotheses, focusing on the situation in which the proportion of false null hypotheses is very small. We propose a family of methods for establishing lower 100(1−α)% confidence bounds for this proportion, based on the empirical distribution of the p-values of the tests. Methods in this family are then compared in terms of ability to consistently estimate the proportion by letting α→0 as the number of hypothesis tests increases and the proportion decreases. This work is motivated by a signal detection problem that occurs in astronomy.

Citations

• Higher criticism for detecting sparse heterogeneous mixtures
• The statistical theory of the strength of bundles of threads. I
• A stochastic process approach to false discovery control

Cited by

• Estimation and confidence sets for sparse normal mixtures
• Goodness-of-fit tests via phi-divergences
• On the false discovery rate and an asymptotically optimal rejection curve
• Higher Criticism for Large-Scale Inference, Especially for Rare and Weak Effects
• Higher Criticism Tuned Regression For Weak And Sparse Signals
• A grouped, selectively weighted false discovery rate procedure