Intent Recognition and Out-of-Scope Detection using LLMs in Multi-party Conversations

2025/07/29 by Castillo-López, Galo, de Chalendar, Gaël, Semmar, Nasredine
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)

paper · doi:10.48550/arxiv.2507.22289

Abstract

Intent recognition is a fundamental component in task-oriented dialogue systems (TODS). Determining user intents and detecting whether an intent is Out-of-Scope (OOS) is crucial for TODS to provide reliable responses. However, traditional TODS require large amount of annotated data. In this work we propose a hybrid approach to combine BERT and LLMs in zero and few-shot settings to recognize intents and detect OOS utterances. Our approach leverages LLMs generalization power and BERT's computational efficiency in such scenarios. We evaluate our method on multi-party conversation corpora and observe that sharing information from BERT outputs to LLMs leads to system performance improvement.

Citations

• A new approach for fine-tuning sentence transformers for intent classification and out-of-scope detection tasks
• Intent Detection in the Age of LLMs
• Generate then Refine: Data Augmentation for Zero-shot Intent Detection
• MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversations
• Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition
• Multi-party Goal Tracking with LLMs: Comparing Pre-training, Fine-tuning, and Prompt Engineering
• Multi-Tenant Optimization For Few-Shot Task-Oriented FAQ Retrieval
• Efficient Intent Detection with Dual Sentence Encoders
• An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• Intent Detection in the Age of LLMs
• Efficient Out-of-Scope Detection in Dialogue Systems via Uncertainty-Driven LLM Routing
• Efficient Intent-Based Filtering for Multi-Party Conversations Using Knowledge Distillation from LLMs
• Multi-Intent Recognition in Dialogue Understanding: A Comparison Between Smaller Open-Source LLMs
• SLIM: Explicit Slot-Intent Mapping with BERT for Joint Multi-Intent Detection and Slot Filling