2023/12/19 by Volodymyr Kuleshov, Piriyakulkij, Wasu Top, Kevin Ellis +2 · 6 citations
Computer Science · #Artificial Intelligence (cs.AI) #Bayesian Modeling and Causal Inference #Computation and Language (cs.CL) #FOS: Computer and information sciences #Logic, Reasoning, and Knowledge #Machine Learning (cs.LG) #Speech and dialogue systems
paper · pdf · doi:10.48550/arxiv.2312.12009
openalex publication_date 2023/12/19 · openalex created_date 2023/12/22 · openalex updated_date 2026/07/28
Actively inferring user preferences, for example by asking good questions, is important for any human-facing decision-making system. Active inference allows such systems to adapt and personalize themselves to nuanced individual preferences. To enable this ability for instruction-tuned large language models (LLMs), one may prompt them to ask users questions to infer their preferences, transforming the language models into more robust, interactive systems. However, out of the box, these models are not efficient at extracting preferences: the questions they generate are not informative, requiring a high number of user interactions and impeding the usability of the downstream system. In this work, we introduce an inference-time algorithm that helps LLMs quickly infer preferences by using more informative questions. Our algorithm uses a probabilistic model whose conditional distributions are defined by prompting an LLM, and returns questions that optimize expected entropy and expected model change. Results in a simplified interactive web shopping setting with real product items show that an LLM equipped with our entropy reduction algorithm outperforms baselines with the same underlying LLM on task performance while using fewer user interactions.