Learning with Selectively Labeled Data from Multiple Decision-makers

2023/06/13 by Jian Chen, Chen, Jian, Zhehao Li +3
Computer Science · Decision Sciences · Engineering · #FOS: Computer and information sciences #Forecasting Techniques and Applications #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Water resources management and optimization

paper · pdf · doi:10.48550/arxiv.2306.07566

openalex publication_date 2023/06/13 · openalex created_date 2023/06/15 · openalex updated_date 2026/07/28

Abstract

We study the problem of classification with selectively labeled data, whose distribution may differ from the full population due to historical decision-making. We exploit the fact that in many applications historical decisions were made by multiple decision-makers, each with different decision rules. We analyze this setup under a principled instrumental variable (IV) framework and rigorously study the identification of classification risk. We establish conditions for the exact identification of classification risk and derive tight partial identification bounds when exact identification fails. We further propose a unified cost-sensitive learning (UCL) approach to learn classifiers robust to selection bias in both identification settings. Finally, we theoretically and numerically validate the efficacy of our proposed method.

Related

• Learning under selective labels in the presence of expert consistency
• Learning Robust Decision Policies from Observational Data
• Multi-class Classification from Multiple Unlabeled Datasets with Partial Risk Regularization
• Joint empirical risk minimization for instance-dependent positive-unlabeled data
• Optimal Policies for the Homogeneous Selective Labels Problem