Efficiently Identifying Task Groupings for Multi-Task Learning

2021/09/10 by Christopher Fifty, Fifty, Christopher, Ehsan Amid +9 · 20 citations
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification

paper · pdf · doi:10.48550/arxiv.2109.04617

openalex publication_date 2021/09/10 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Multi-task learning can leverage information learned by one task to benefit the training of other tasks. Despite this capacity, naively training all tasks together in one model often degrades performance, and exhaustively searching through combinations of task groupings can be prohibitively expensive. As a result, efficiently identifying the tasks that would benefit from training together remains a challenging design question without a clear solution. In this paper, we suggest an approach to select which tasks should train together in multi-task learning models. Our method determines task groupings in a single run by training all tasks together and quantifying the effect to which one task's gradient would affect another task's loss. On the large-scale Taskonomy computer vision dataset, we find this method can decrease test loss by 10.0% compared to simply training all tasks together while operating 11.6 times faster than a state-of-the-art task grouping method.

Citations

• An Overview of Multi-Task Learning in Deep Neural Networks
• A Survey on Multi-Task Learning
• Recasting Gradient-Based Meta-Learning as Hierarchical Bayes
• On First-Order Meta-Learning Algorithms
• Which Tasks Should Be Learned Together in Multi-task Learning?
• Chameleon: Learning Model Initializations Across Tasks With Different Schemas
• Understanding and Improving Information Transfer in Multi-Task Learning
• Regularizing Deep Multi-Task Networks using Orthogonal Gradients
• Gradient Surgery for Multi-Task Learning
• MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale

Cited by

• Simplifying Multi-Task Architectures Through Task-Specific Normalization
• Meta Lattice: Model Space Redesign for Cost-Effective Industry-Scale Ads Recommendations
• Efficiently Learning Branching Networks for Multitask Algorithmic Reasoning
• Understanding, Accelerating, and Improving MeanFlow Training
• FSDAM: Few-Shot Driving Attention Modeling via Vision-Language Coupling
• Root Cause Analysis for Microservice Systems via Cascaded Conditional Learning with Hypergraphs
• Heterogeneous Adversarial Play in Interactive Environments
• Quantifying Dataset Similarity to Guide Transfer Learning
• Neuroplasticity-inspired dynamic ANNs for multi-task demand forecasting
• Monotonic Transformation Invariant Multi-task Learning
• Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysis
• Parameter-Efficient Multi-Task Learning via Progressive Task-Specific Adaptation
• Graph Coloring for Multi-Task Learning
• SMILE: A Super-resolution Guided Multi-task Learning Method for Hyperspectral Unmixing
• GCond: Gradient Conflict Resolution via Accumulation-based Stabilization for Large-Scale Multi-Task Learning
• DivMerge: A divergence-based model merging method for multi-tasking
• ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning
• Linear-Time Demonstration Selection for In-Context Learning via Gradient Estimation
• Cross-Cancer Knowledge Transfer in WSI-based Prognosis Prediction
• Batching-Aware Joint Model Onloading and Offloading for Hierarchical Multi-Task Inference

Related

• Towards Principled Task Grouping for Multi-Task Learning
• Which Tasks Should Be Learned Together in Multi-task Learning?
• Task Grouping for Automated Multi-Task Machine Learning via Task Affinity Prediction
• Revisit the Imbalance Optimization in Multi-task Learning: An Experimental Analysis
• Selective Task Group Updates for Multi-Task Optimization