Bag of Tricks for Efficient Text Classification
2016/07/06 by Armand Joulin, Edouard Grave, Joulin, Armand +5 · 3 voices · 77 citations
#cs.CL
paper · pdf · doi:10.48550/arxiv.1607.01759
Abstract
This paper explores a simple and efficient baseline for text classification. Our experiments show that our fast text classifier fastText is often on par with deep learning classifiers in terms of accuracy, and many orders of magnitude faster for training and evaluation. We can train fastText on more than one billion words in less than ten minutes using a standard multicore~CPU, and classify half a million sentences among~312K classes in less than a minute.
Cited by
- Glyce: Glyph-vectors for Chinese Character Representations
- Language Identification via Compositional Data Analysis: A Linear-Time Classifier Based on Log-Ratio Geometry
- Pretraining Data Can Be Poisoned through Computational Propaganda
- Parallel by design? Meaning and grammar in single-stream and dual-stream neural network architectures
- Subliminal Effects in Your Data: A General Mechanism via Log-Linearity
- CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data
- GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models
- DataRater: Meta-Learned Dataset Curation
- SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving
- Influence of Prompt Engineering on Small Language Models for Guarded Query Routing
- CuraWeb: Joint Optimization of Quality, Redundancy, and Diversity for Web-Scale Pretraining Data
- A Study of Crosslinguistic Influence in Language Models
- MiniLingua: A Small Open-Source LLM for European Languages
- PolyLingua: Margin-based Inter-class Transformer for Robust Cross-domain Language Detection
- Luxical: High-Speed Lexical-Dense Text Embeddings
- Persian-Phi: Efficient Cross-Lingual Adaptation of Compact LLMs via Curriculum Learning
- What Language is This? Ask Your Tokenizer
- Cross-Lingual Prompt Steerability: Towards Accurate and Robust LLM Behavior across Languages
- A transfer learning approach for automatic conflicts detection in software requirement sentence pairs based on dual encoders
- Winning with Less for Low Resource Languages: Advantage of Cross-Lingual EnglishPersian Argument Mining Model over LLM Augmentation
- LLMs for Low-Resource Dialect Translation Using Context-Aware Prompting: A Case Study on Sylheti
- How Learning Rate Decay Wastes Your Best Data in Curriculum-Based LLM Pretraining
- MTikGuard System: A Transformer-Based Multimodal System for Child-Safe Content Moderation on TikTok
- Blu-WERP (Web Extraction and Refinement Pipeline): A Scalable Pipeline for Preprocessing Large Language Model Datasets
- Attention-Guided Feature Fusion (AGFF) Model for Integrating Statistical and Semantic Features in News Text Classification
- Standardising the NLP Workflow: A Framework for Reproducible Linguistic Analysis
- How Do Data Owners Say No? A Case Study of Data Consent Mechanisms in Web-Scraped Vision-Language AI Training Datasets
- Characterizing AI Manipulation Risks in Brazilian YouTube Climate Discourse
- ManufactuBERT: Efficient Continual Pretraining for Manufacturing
- Penetrating the Hostile: Detecting DeFi Protocol Exploits through Cross-Contract Analysis
- Language Modeling With Factorization Memory
- Modular Linear Tokenization (MLT)
- CHALIS: A Challenge Dataset for Language Identification in Difficult Scenarios
- A Bitter Lesson for Data Filtering
- The Moltbook Files: A Harmless Slopocalypse or Humanity's Last Experiment
- SHA-256 Infused Embedding-Driven Generative Modeling of High-Energy Molecules in Low-Data Regimes
- Streamlining business functions in official statistical production with Machine Learning
- A Survey on LLM Mid-Training
- Iterative Layer Pruning for Efficient Translation Inference
- Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset
- The Art of Asking: Multilingual Prompt Optimization for Synthetic Data
- ReasonIF: Large Reasoning Models Fail to Follow Instructions During Reasoning
- BiMax: Bidirectional MaxSim Score for Document-Level Alignment
- Train a Unified Multimodal Data Quality Classifier with Synthetic Data
- The German Commons - 154 Billion Tokens of Openly Licensed Text for German Language Models
- Tahakom LLM Guidelines and Recipes: From Pre-training Data to an Arabic LLM
- Tandem Training for Language Models
- A Multi-lingual Dataset of Classified Paragraphs from Open Access Scientific Publications
- From Birdwatch to Community Notes, from Twitter to X: four years of community-based content moderation
- Domain-Adapted Pre-trained Language Models for Implicit Information Extraction in Crash Narratives
- Investigating Thematic Patterns and User Preferences in LLM Interactions using BERTopic
- Language Lives in Sparse Dimensions: Toward Interpretable and Efficient Multilingual Control for Large Language Models
- Mining the Mind: What 100M Beliefs Reveal About Frontier LLM Knowledge
- Scaling Laws Revisited: Modeling the Role of Data Quality in Language Model Pretraining
- Prior-based Noisy Text Data Filtering: Fast and Strong Alternative For Perplexity
- DIVERS-Bench: Evaluating Language Identification Across Domain Shifts and Code-Switching
- CommonForms: A Large, Diverse Dataset for Form Field Detection
- Patent Language Model Pretraining with ModernBERT
- Byte by Byte: Unmasking Browser Fingerprinting at the Function Level Using V8 Bytecode Transformers
- Boosting Data Utilization for Multilingual Dense Retrieval
- Llama-GENBA-10B: A Trilingual Large Language Model for German, English and Bavarian
- PLaMo 2 Technical Report
- Explicit and Implicit Data Augmentation for Social Event Detection
- Efficient Item ID Generation for Large-Scale LLM-based Recommendation
- Expanding the WMT24++ Benchmark with Rumantsch Grischun, Sursilvan, Sutsilvan, Surmiran, Puter, and Vallader
- Abex-rat: Synergizing Abstractive Augmentation and Adversarial Training for Classification of Occupational Accident Reports
- SyGra: A Unified Graph-Based Framework for Scalable Generation, Quality Tagging, and Management of Synthetic Data
- Improving LLMs for Machine Translation Using Synthetic Preference Data
- Comparing energy consumption and accuracy in text classification inference
- Hierarchical knowledge guided fault intensity diagnosis of complex industrial systems
- Incorporating Legal Logic into Deep Learning: An Intelligent Approach to Probation Prediction
- Llama-3.1-FoundationAI-SecurityLLM-8B-Instruct Technical Report
- Language Arithmetics: Towards Systematic Language Neuron Identification and Manipulation
- VN-MTEB: Vietnamese Massive Text Embedding Benchmark
- FastText [wikipedia]
- Tomáš Mikolov [wikipedia]
- Word2vec [wikipedia]
Discussions
Related