Conditional Testing based on Localized Conformal p-values

2024/09/25 by Wu, Xiaoyang, Lu, Lin, Wang, Zhaojun +1 · 2 citations
#FOS: Computer and information sciences #Machine Learning (stat.ML) #Methodology (stat.ME)

paper · doi:10.48550/arxiv.2409.16829

Abstract

In this paper, we address conditional testing problems through the conformal inference framework. We define the localized conformal p-values by inverting prediction intervals and prove their theoretical properties. These defined p-values are then applied to several conditional testing problems to illustrate their practicality. Firstly, we propose a conditional outlier detection procedure to test for outliers in the conditional distribution with finite-sample false discovery rate (FDR) control. We also introduce a novel conditional label screening problem with the goal of screening multivariate response variables and propose a screening procedure to control the family-wise error rate (FWER). Finally, we consider the two-sample conditional distribution test and define a weighted U-statistic through the aggregation of localized p-values. Numerical simulations and real-data examples validate the superior performance of our proposed strategies.

Cited by

• Selective Labeling with False Discovery Rate Control
• Theoretical guarantees for change localization using conformal p-values

Related

• Integrative conformal p-values for powerful out-of-distribution testing with labeled outliers
• Probabilistic Conformal Prediction with Approximate Conditional Validity
• Conformal prediction with local weights: randomization enables local guarantees
• Model-free selective inference under covariate shift via weighted conformal p-values
• False Discovery Rate Control For Structured Multiple Testing: Asymmetric Rules And Conformal Q-values