Learning the Truth From Only One Side of the Story

2020/06/08 by Heinrich Jiang, Jiang, Heinrich, Qijia Jiang +3
Computer Science · Social Sciences · #Ethics and Social Impacts of AI #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2006.04858

openalex publication_date 2020/06/08 · openalex created_date 2020/06/19 · openalex updated_date 2026/07/28

Abstract

Learning under one-sided feedback (i.e., where we only observe the labels for examples we predicted positively on) is a fundamental problem in machine learning -- applications include lending and recommendation systems. Despite this, there has been surprisingly little progress made in ways to mitigate the effects of the sampling bias that arises. We focus on generalized linear models and show that without adjusting for this sampling bias, the model may converge suboptimally or even fail to converge to the optimal solution. We propose an adaptive approach that comes with theoretical guarantees and show that it outperforms several existing methods empirically. Our method leverages variance estimation techniques to efficiently learn under uncertainty, offering a more principled alternative compared to existing approaches.

Citations

• Contextual Bandit Algorithms with Supervised Learning Guarantees
• Equality of Opportunity in Supervised Learning

Related

• Fine-Grained Dynamic Framework for Bias-Variance Joint Optimization on Data Missing Not at Random
• Probably Approximately Precision and Recall Learning
• Machine Learning. The Science of Selection under Uncertainty
• Mitigating Label Bias in Machine Learning: Fairness through Confident Learning
• A Regret-Variance Trade-Off in Online Learning