2024/02/17 by Vijay Keswani, Keswani, Vijay, Anay Mehrotra +3 · 1 citation
Economics, Econometrics and Finance · Social Sciences · #Artificial Intelligence (cs.AI) #Cambodian History and Society #Computers and Society (cs.CY) #FOS: Computer and information sciences #Law, Economics, and Judicial Systems #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Qualitative Comparative Analysis Research
paper · pdf · doi:10.48550/arxiv.2402.11338
openalex publication_date 2024/02/17 · openalex created_date 2024/02/22 · openalex updated_date 2026/07/28
In many predictive contexts (e.g., credit lending), true outcomes are only observed for samples that were positively classified in the past. These past observations, in turn, form training datasets for classifiers that make future predictions. However, such training datasets lack information about the outcomes of samples that were (incorrectly) negatively classified in the past and can lead to erroneous classifiers. We present an approach that trains a classifier using available data and comes with a family of exploration strategies to collect outcome data about subpopulations that otherwise would have been ignored. For any exploration strategy, the approach comes with guarantees that (1) all sub-populations are explored, (2) the fraction of false positives is bounded, and (3) the trained classifier converges to a ``desired'' classifier. The right exploration strategy is context-dependent; it can be chosen to improve learning guarantees and encode context-specific group fairness properties. Evaluation on real-world datasets shows that this approach consistently boosts the quality of collected outcome data and improves the fraction of true positives for all groups, with only a small reduction in predictive utility.