TACOS: Open Tagging and Comparative Scoring for Instruction Fine-Tuning Data Selection

2025/07/04 by X. Q. He, Hao Yu, He, Xixiang +11
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Educational Technology and Assessment #FOS: Computer and information sciences #Natural Language Processing Techniques

paper · pdf · doi:10.48550/arxiv.2507.03673

openalex publication_date 2025/07/04 · openalex created_date 2025/10/20 · openalex updated_date 2026/07/30

Abstract

Instruction Fine-Tuning (IFT) is crucial for aligning large language models (LLMs) with human preferences, and selecting a small yet representative subset from massive data significantly facilitates IFT in terms of both efficiency and effectiveness. Nevertheless, existing approaches suffer from two limitations: the use of simple heuristics restricts data diversity, while the singleton data quality evaluation accounts for inconsistent criteria between independent samples. To address the issues, we present TACOS, an innovative method that integrates Open Tagging and Comparative Scoring for IFT data selection. To capture data diversity, we leverage LLMs to assign open-domain tags to human queries, followed by a normalization stage to denoise the open tags and enable efficient clustering. Additionally, we suggest a comparative scoring method that allows the relative quality evaluation of samples within a cluster, avoiding inconsistent criteria seen in singleton-based evaluations. Extensive experiments across diverse datasets and LLM architectures demonstrate that TACOS outperforms existing approaches by a large margin. Notably, it achieves superior instruction-following performance on MT-Bench and ranks 1st among LLaMA2-7B-Based models on AlpacaEval 2.0, illustrating its efficacy for IFT data selection.

Citations

• ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools
• LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models
• Long Is More for Alignment: A Simple but Tough-to-Beat Baseline for Instruction Fine-Tuning
• What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning
• MoDS: Model-oriented Data Selection for Instruction Tuning
• Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2
• Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning
• Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks
• InstructCoder: Instruction Tuning Large Language Models for Code Editing
• Understanding the Effects of RLHF on LLM Generalisation and Diversity
• Mistral 7B
• NEFTune: Noisy Embeddings Improve Instruction Finetuning
• Improving Translation Faithfulness of Large Language Models via Augmenting Instructions
• From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning
• #InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Models
• A Preliminary Study of the Intrinsic Relationship between Complexity and Alignment
• Llama 2: Open Foundation and Fine-Tuned Chat Models
• AlpaGasus: Training A Better Alpaca with Fewer Data
• Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
• Orca: Progressive Learning from Complex Explanation Traces of GPT-4
• LIMA: Less Is More for Alignment
• Maybe Only 0.5% Data is Needed: A Preliminary Exploration of Low Training Data Instruction Tuning
• GPT-4 Technical Report
• The Flan Collection: Designing Data and Methods for Effective Instruction Tuning
• Self-Instruct: Aligning Language Models with Self-Generated Instructions
• Demystifying Prompts in Language Models via Perplexity Estimation
• ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation
• Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks
• Training language models to follow instructions with human feedback
• Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration
• Finetuned Language Models Are Zero-Shot Learners
• Instruction Mining: Instruction Data Selection for Tuning Large Language Models
• WizardLM: Empowering large pre-trained language models to follow complex instructions

Related

• Rethinking Data Selection at Scale: Random Selection is Almost All You Need
• T-SHIRT: Token-Selective Hierarchical Data Selection for Instruction Tuning
• Improving Data Efficiency via Curating LLM-Driven Rating Systems
• Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models
• MIG: Automatic Data Selection for Instruction Tuning by Maximizing Information Gain in Semantic Space