Disentangling Confidence Score Distribution for Out-of-Domain Intent Detection with Energy-Based Learning

2022/10/17 by Yanan Wu, Zhiyuan Zeng, Wu, Yanan +11
Computer Science · #Computation and Language (cs.CL) #Data Stream Mining Techniques #FOS: Computer and information sciences #Spam and Phishing Detection #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2210.08830

openalex publication_date 2022/10/17 · openalex created_date 2022/10/20 · openalex updated_date 2026/07/28

Abstract

Detecting Out-of-Domain (OOD) or unknown intents from user queries is essential in a task-oriented dialog system. Traditional softmax-based confidence scores are susceptible to the overconfidence issue. In this paper, we propose a simple but strong energy-based score function to detect OOD where the energy scores of OOD samples are higher than IND samples. Further, given a small set of labeled OOD samples, we introduce an energy-based margin objective for supervised OOD detection to explicitly distinguish OOD samples from INDs. Comprehensive experiments and analysis prove our method helps disentangle confidence score distributions of IND and OOD data.\footnoteOur code is available at \urlhttps://github.com/pris-nlp/EMNLP2022-energy_for_OOD/.

Related

• Energy-based Unknown Intent Detection with Data Manipulation
• UniNL: Aligning Representation Learning with Scoring Function for OOD Detection via Unified Neighborhood Learning
• Estimating Soft Labels for Out-of-Domain Intent Detection
• The Open-World Lottery Ticket Hypothesis for OOD Intent Classification
• Distribution Calibration for Out-of-Domain Detection with Bayesian Approximation