Let Me At Least Learn What You Really Like: Dealing With Noisy Humans When Learning Preferences

2020/02/15 by Sriram Gopalakrishnan, Gopalakrishnan, Sriram, Utkarsh Soni +1
Computer Science · #Computability, Logic, AI Algorithms #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2002.06288

openalex publication_date 2020/02/15 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Learning the preferences of a human improves the quality of the interaction with the human. The number of queries available to learn preferences maybe limited especially when interacting with a human, and so active learning is a must. One approach to active learning is to use uncertainty sampling to decide the informativeness of a query. In this paper, we propose a modification to uncertainty sampling which uses the expected output value to help speed up learning of preferences. We compare our approach with the uncertainty sampling baseline, as well as conduct an ablation study to test the validity of each component of our approach.

Related

• Active Preference Inference using Language Models and Probabilistic Reasoning
• On weighted uncertainty sampling in active learning
• Enhancing Preference-based Linear Bandits via Human Response Time
• Uncertainty-Penalized Direct Preference Optimization
• Preference Construction: A Bayesian Interactive Preference Elicitation Framework Based on Monte Carlo Tree Search