2025/11/17 by Rajesh Karmakar, Karmakar, Rajesh, Ruth Heller +3
Computer Science · Decision Sciences · Mathematics · #FOS: Computer and information sciences #Methodology (stat.ME) #Psychometric Methodologies and Testing #Software Testing and Debugging Techniques #Statistical Methods in Clinical Trials
paper · pdf · doi:10.48550/arxiv.2511.13624
openalex publication_date 2025/11/17 · openalex created_date 2025/11/19 · openalex updated_date 2026/07/28
We seek to design novel multiple testing procedures, which take into account a relevant notion of ''power'' or true discovery on the one hand, and allow computationally efficient test design and application on the other. Towards this end we characterize the optimal procedures that strongly control the family-wise error rate, for a range of power objectives measuring the success of multiple testing procedures in making true individual discoveries, and under a reasonable set of assumptions. While we cannot generally find these optimal solutions in practice, we propose the bottom-up approach, which constructs consonant closed testing procedures, while taking into account the overall power objective in designing the tests on every level of the closed testing hierarchy. This leads to a general recipe, yielding novel procedures which are computationally practical and demonstrate substantially improved power in both simulations and a real data study, compared to existing procedures.