SDO: Structure-Aware Data Organization for Efficient LLM Post-Training

2026/07/29 by Jinliang Gao, Ning Yang, Hai Wang +2
Computer Science · #cs.LG

paper · pdf

9 pages, 5 figures, 5 tables

arxiv created 2026/07/29 · arxiv updated 2026/07/31

Abstract

Post-training of large language models is expensive, and existing efficiency improvements mainly focus on selecting informative samples or designing training schedules. However, data organization itself is usually treated as a static preprocessing step: embedding-based grouping methods construct fixed partitions before training and cannot adapt to the evolving sample exposure during optimization. As a result, all samples receive similar exposure despite their different optimization needs, leading to redundant updates for some samples while leaving others under-optimized. To address this problem, we propose SDO (Structure-Aware Data Organization), a plug-and-play data organization framework with an exposure-driven feedback mechanism that organizes mini-batch composition and sample exposure according to representation-space structure. SDO operates epoch by epoch on frozen external embeddings, avoiding model warm-up training overhead: within each epoch, locality-aware batching forms coherent mini-batches via KNN neighborhood traversal; across epochs, exposure-balanced scheduling records per-sample participation and reduces the sampling probability of over-exposed samples to preserve long-term coverage. Across SFT, DPO, and GRPO, SDO accelerates convergence, with the largest gains observed in the early-to-mid phase, producing more coherent gradients and more balanced accuracy across question types without permanently excluding training samples.

Citations

• A Survey on Efficient Large Language Model Training: From Data-centric Perspectives
• Prompt Curriculum Learning for Efficient LLM Post-Training
• LESS: Selecting Influential Data for Targeted Instruction Tuning
• BioXP-0.5B: Explainable Medical-AI via RL-GRPO
• What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning
• AlpaGasus: Training A Better Alpaca with Fewer Data
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model
• Data Selection for Language Models via Importance Resampling
• Beyond neural scaling laws: beating power law scaling via data pruning
• Training language models to follow instructions with human feedback
• Training Verifiers to Solve Math Word Problems
• Scaling Laws for Neural Language Models
• Generalization through Memorization: Nearest Neighbor Language Models
• Selection via Proxy: Efficient Data Selection for Deep Learning
• Not All Samples Are Created Equal: Deep Learning with Importance Sampling
• Don't Decay the Learning Rate, Increase the Batch Size
• Minimizing Finite Sums with the Stochastic Average Gradient
• UltraFeedback: Boosting Language Models with Scaled AI Feedback
• Automatic Pair Construction for Contrastive Post-training

Related

• Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM
• Middo: Model-Informed Dynamic Data Optimization for Enhanced LLM Fine-Tuning via Closed-Loop Learning
• Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs
• D4: Improving LLM Pretraining via Document De-Duplication and Diversification
• Take the Bull by the Horns: Hard Sample-Reweighted Continual Training Improves LLM Generalization