Small Language Models in the Real World: Insights from Industrial Text Classification

2025/05/21 by Lujun Li, Li, Lujun, Lama Sleem +7
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning and Data Classification #Text and Document Classification Technologies #Topic Modeling

paper · pdf · doi:10.48550/arxiv.2505.16078

openalex publication_date 2025/05/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

With the emergence of ChatGPT, Transformer models have significantly advanced text classification and related tasks. Decoder-only models such as Llama exhibit strong performance and flexibility, yet they suffer from inefficiency on inference due to token-by-token generation, and their effectiveness in text classification tasks heavily depends on prompt quality. Moreover, their substantial GPU resource requirements often limit widespread adoption. Thus, the question of whether smaller language models are capable of effectively handling text classification tasks emerges as a topic of significant interest. However, the selection of appropriate models and methodologies remains largely underexplored. In this paper, we conduct a comprehensive evaluation of prompt engineering and supervised fine-tuning methods for transformer-based text classification. Specifically, we focus on practical industrial scenarios, including email classification, legal document categorization, and the classification of extremely long academic texts. We examine the strengths and limitations of smaller models, with particular attention to both their performance and their efficiency in Video Random-Access Memory (VRAM) utilization, thereby providing valuable insights for the local deployment and application of compact models in industrial settings.

Citations

• Chain of Draft: Thinking Faster by Writing Less
• A Comparative Study of Clinical ModernBERT and BioMedical ModernBERT on the DDXPlus Dataset
• The Ultimate Guide to Fine-Tuning LLMs from Basics to Breakthroughs: An Exhaustive Review of Technologies, Research, Best Practices, Applied Research Challenges and Opportunities
• Gemma 2: Improving Open Language Models at a Practical Size
• How do Large Language Models Handle Multilingualism?
• LLaMA: Open and Efficient Foundation Language Models
• How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation
• Self-Consistency Improves Chain of Thought Reasoning in Language Models
• BNAI, NO-TOKEN, and MIND-UNITY: Pillars of a Systemic Revolution in Artificial Intelligence
• Which Student is Best? A Comprehensive Knowledge Distillation Exam for Task-Specific BERT Models
• NER-BERT: A Pre-trained Model for Low-Resource Entity Tagging
• The Power of Scale for Parameter-Efficient Prompt Tuning
• Language Models are Few-Shot Learners
• Longformer: The Long-Document Transformer
• BART: Denoising Sequence-to-Sequence Pre-training for Natural Language\n Generation, Translation, and Comprehension
• Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• RoBERTa: A Robustly Optimized BERT Pretraining Approach
• Large-Scale Multi-Label Text Classification on EU Legislation
• Generative and Discriminative Text Classification with Recurrent Neural Networks
• Convolutional Neural Networks for Sentence Classification
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• Prefix-Tuning: Optimizing Continuous Prompts for Generation
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Related

• Using Language Models on Low-end Hardware
• Fine-Tuning Causal LLMs for Text Classification: Embedding-Based vs. Instruction-Based Approaches
• Large Language Models For Text Classification: Case Study And Comprehensive Review
• Small Language Models: Survey, Measurements, and Insights
• Open, Closed, or Small Language Models for Text Classification?