2025/04/15 by Michał Łukasik, Lin Chen, Lukasik, Michal +17 · 2 citations
Business, Management and Accounting · Decision Sciences · Economics, Econometrics and Finance · #Artificial Intelligence (cs.AI) #Consumer Market Behavior and Pricing #FOS: Computer and information sciences #Game Theory and Voting Systems #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multi-Criteria Decision Making
paper · pdf · doi:10.48550/arxiv.2504.11284
openalex publication_date 2025/04/15 · openalex created_date 2025/10/14 · openalex updated_date 2026/07/28
Bipartite ranking is a fundamental supervised learning problem, with the goal of learning a ranking over instances with maximal Area Under the ROC Curve (AUC) against a single binary target label. However, one may often observe multiple binary target labels, e.g., from distinct human annotators. How can one synthesize such labels into a single coherent ranking? In this work, we formally analyze two approaches to this problem -- loss aggregation and label aggregation -- by characterizing their Bayes-optimal solutions. We show that while both approaches can yield Pareto-optimal solutions, loss aggregation can exhibit label dictatorship: one can inadvertently (and undesirably) favor one label over others. This suggests that label aggregation can be preferable to loss aggregation, which we empirically verify.