Human-In-The-Loop Learning of Qualitative Preference Models

2019/09/19 by Joseph H. Allen, Joseph Allen, Allen, Joseph +4
Computer Science · #Artificial Intelligence (cs.AI) #Constraint Satisfaction and Optimization #Data Management and Algorithms #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Semantic Web and Ontologies #cs.AI #cs.HC

paper · pdf · doi:10.48550/arxiv.1909.09064

Published in the Proceedings of the 32nd International Florida Artificial Intelligence Research Society Conference, 2019

arxiv created 2019/09/19 · openalex publication_date 2019/09/19 · arxiv updated 2019/09/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

In this work, we present a novel human-in-the-loop framework to help the human user understand the decision making process that involves choosing preferred options. We focus on qualitative preference models over alternatives from combinatorial domains. This framework is interactive: the user provides her behavioral data to the framework, and the framework explains the learned model to the user. It is iterative: the framework collects feedback on the learned model from the user and tries to improve it accordingly till the user terminates the iteration. In order to communicate the learned preference model to the user, we develop visualization of intuitive and explainable graphic models, such as lexicographic preference trees and forests, and conditional preference networks. To this end, we discuss key aspects of our framework for lexicographic preference models.

Related

• Learning a Canonical Basis of Human Preferences from Binary Ratings
• LRHP: Learning Representations for Human Preferences via Preference Pairs
• Preference Transformer: Modeling Human Preferences using Transformers for RL
• Comparative Explanations: Explanation Guided Decision Making for Human-in-the-Loop Preference Selection
• ICPL: Few-shot In-context Preference Learning via LLMs