Learn2Mix: Training Neural Networks Using Adaptive Data Integration

2024/12/21 by Shyam Venkatasubramanian, Venkatasubramanian, Shyam, Vahid Tarokh +1 · 1 citation
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications

paper · pdf · doi:10.48550/arxiv.2412.16482

openalex publication_date 2024/12/21 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Accelerating model convergence in resource-constrained environments is essential for fast and efficient neural network training. This work presents learn2mix, a new training strategy that adaptively adjusts class proportions within batches, focusing on classes with higher error rates. Unlike classical training methods that use static class proportions, learn2mix continually adapts class proportions during training, leading to faster convergence. Empirical evaluations on benchmark datasets show that neural networks trained with learn2mix converge faster than those trained with existing approaches, achieving improved results for classification, regression, and reconstruction tasks under limited training resources and with imbalanced classes. Our empirical findings are supported by theoretical analysis.

Cited by

• Optimizing Class Distributions for Bias-Aware Multi-Class Learning

Related

• Simplifying Neural Network Training Under Class Imbalance
• AdaBatch: Adaptive Batch Sizes for Training Deep Neural Networks
• DIVEBATCH: Accelerating Model Training Through Gradient-Diversity Aware Batch Size Adaptation
• Accelerated Training via Incrementally Growing Neural Networks using Variance Transfer and Learning Rate Adaptation
• Adaptive Optimization Algorithms for Machine Learning