Training-Free versus Training-Based Intent Classification in LLMs: Accuracy, Robustness, and Failure Modes

2026/08/03 by Nan Chen, Zhouhao Yang, Soufiane Hayou
Computer Science · #cs.CL

paper · pdf

Accepted at the Conference on Language Modeling (COLM 2026)

arxiv created 2026/08/03 · arxiv updated 2026/08/04

Abstract

Intent classification in Large Language Models (LLMs) involves categorizing user prompts into predefined classes. For instance, given a user prompt, the system must determine whether it primarily concerns mathematics, coding, or general text processing. Such classification enables routing prompts to specialized models optimized for specific domains, improving both accuracy and computational efficiency. In this work, we conduct a systematic study comparing training-free vs training-based approaches for intent classification. For this purpose, we consider two lightweight, training-free methods based on statistics of internal representations and compare them against MLP classifiers and linear probes. Our comprehensive empirical evaluation reveals that 1) Both training-free and training-based methods saturate easy benchmarks (mathematics vs. coding vs. natural language), 2) Training-based classifiers have an advantage on harder classification tasks (e.g. Java vs Python), and 3) Training-free methods are generally more robust to mixed-intent and adversarial prompts.

Citations

• Efficient Training-Free Online Routing for High-Volume Multi-LLM Serving
• PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models
• Qwen3 Technical Report
• SpeCache: Speculative Key-Value Caching for Efficient Generation of LLMs
• Universal Model Routing for Efficient LLM Inference
• CARROT: A Cost Aware Rate Optimal Router
• LLMs are Also Effective Embedding Models: An In-depth Overview
• When Text Embedding Meets Large Language Model: A Comprehensive Survey
• IntentGPT: Few-shot Intent Discovery with Large Language Models
• A Unified Approach to Routing and Cascading for LLMs
• GraphRouter: A Graph-based Router for LLM Selections
• Intent Detection in the Age of LLMs
• Eagle: Efficient Training-Free Router for Multi-LLM Inference
• LLMs Will Always Hallucinate, and We Need to Live With This
• Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?
• TensorOpera Router: A Multi-Model Router for Efficient LLM Inference
• The Llama 3 Herd of Models
• DeepSeek-Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence
• Scaling and evaluating sparse autoencoders
• NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models
• LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders
• Generative Representational Instruction Tuning
• DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence
• Improving Text Embeddings with Large Language Models
• Steering Llama 2 via Contrastive Activation Addition
• Magicoder: Empowering Code Generation with OSS-Instruct
• Large Language Models Are Zero-Shot Text Classifiers
• AutoMix: Automatically Mixing Language Models
• Fine-Tuning LLaMA for Multi-Stage Text Retrieval
• Large Language Model Cascades with Mixture of Thoughts Representations for Cost-efficient Reasoning
• Fusing Models with Complementary Expertise
• Sparse Autoencoders Find Highly Interpretable Features in Language Models
• Tryage: Real-time, intelligent Routing of User Prompts to Large Language Models
• Scaling Sentence Embeddings with Large Language Models
• LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusion
• Let's Verify Step by Step
• GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance
• A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity
• PAL: Program-aided Language Models
• Governance Architecture for Neural Network Superposition: A Structural Solution to Hallucination via Routing and Interference Filtering
• SGPT: GPT Sentence Embeddings for Semantic Search
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Text and Code Embeddings by Contrastive Pre-Training
• Training Verifiers to Solve Math Word Problems
• Datasets: A Community Library for Natural Language Processing
• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
• Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing
• Evaluating Large Language Models Trained on Code
• Measuring Mathematical Problem Solving With the MATH Dataset
• Measuring Massive Multitask Language Understanding
• Aligning AI With Shared Human Values
• DIET: Lightweight Language Understanding for Dialogue Systems
• Efficient Intent Detection with Dual Sentence Encoders
• Fast Transformer Decoding: One Write-Head is All You Need
• An Evaluation Dataset for Intent Classification and Out-of-Scope\n Prediction
• BERT for Joint Intent Classification and Slot Filling
• Rasa: Open Source Language Understanding and Dialogue Management
• Understanding intermediate layers using linear classifier probes
• Palu: Compressing KV-Cache with Low-Rank Projection