A Survey on Multi-Task Learning

2021/03/31 by Yu Zhang, Qiang Yang · 124 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Machine Learning and Algorithms #Machine Learning and ELM

paper · doi:10.1109/tkde.2021.3070203

openalex publication_date 2021/03/31 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/31

Abstract

Multi-Task Learning (MTL) is a learning paradigm in machine learning and its aim is to leverage useful information contained in multiple related tasks to help improve the generalization performance of all the tasks. In this paper, we give a survey for MTL from the perspective of algorithmic modeling, applications and theoretical analyses. For algorithmic modeling, we give a definition of MTL and then classify different MTL algorithms into five categories, including feature learning approach, low-rank approach, task clustering approach, task relation learning approach and decomposition approach as well as discussing the characteristics of each approach. In order to improve the performance of learning tasks further, MTL can be combined with other learning paradigms including semi-supervised learning, active learning, unsupervised learning, reinforcement learning, multi-view learning and graphical models. When the number of tasks is large or the data dimensionality is high, we review online, parallel and distributed MTL models as well as dimensionality reduction and feature hashing to reveal their computational and storage advantages. Many real-world applications use MTL to boost their performance and we review representative works in this paper. Finally, we present theoretical analyses and discuss several future directions for MTL.

Citations

• Excess risk bounds for multitask learning with trace norm regularization
• A Review on Multi-Label Learning Algorithms
• Cross-Domain Multitask Learning with Latent Probit Models
• A Survey on Multi-Task Learning
• Bootstrap Latent-Predictive Representations for Multitask Reinforcement Learning

Cited by

• MEL: Efficient Multi-Task Evolutionary Learning for High-Dimensional Feature Selection
• Unicorn: A Unified Multi-Tasking Matching Model
• Parrot: A Training Pipeline Enhances Both Program CoT and Natural Language CoT for Reasoning
• Generating Auxiliary Tasks with Reinforcement Learning
• CLEANet: Robust and Efficient Anomaly Detection in Contaminated Multivariate Time Series
• NTKMTL: Mitigating Task Imbalance in Multi-Task Learning from Neural Tangent Kernel Perspective
• Multi-Task Learning in Natural Language Processing: An Overview
• Multi-Task Learning with Feature-Similarity Laplacian Graphs for Predicting Alzheimer's Disease Progression
• A Generic Machine Learning Framework for Radio Frequency Fingerprinting
• An End-to-End Room Geometry Constrained Depth Estimation Framework for Indoor Panorama Images
• MetaVLA: Unified Meta Co-training For Efficient Embodied Adaption
• Rethinking Parameter Sharing for LLM Fine-Tuning with Multiple LoRAs
• Neuroplasticity-inspired dynamic ANNs for multi-task demand forecasting
• United We Stand: Towards End-to-End Log-based Fault Diagnosis via Interactive Multi-Task Learning
• ADAPT: Lightweight, Long-Range Machine Learning Force Fields Without Graphs
• NeuSO: Neural Optimizer for Subgraph Queries
• Same Graph Cross-Task Transfer in GNNs: Protocols and Predictors
• Effect of Demographic Bias on Skin Lesion Classification
• FlowDrive: Energy Flow Field for End-to-End Autonomous Driving
• MEJO: MLLM-Engaged Surgical Triplet Recognition via Inter- and Intra-Task Joint Optimization
• Spatially distributed antecedent rainfall thresholds for landslide occurrence: a multitask machine learning modelling approach
• Modular PE-Structured Learning for Cross-Task Wireless Communications
• AnalysisGNN: Unified Music Analysis with Graph Neural Networks
• Robust and Adaptive Spectral Method for Representation Multi-Task Learning with Contamination
• Enhanced Sampling in the Age of Machine Learning: Algorithms and Applications
• Learning to Route: Per-Sample Adaptive Routing for Multimodal Multitask Prediction
• Multi-Faceted Hierarchical Multi-Task Learning for a Large Number of Tasks with Multi-dimensional Relations
• Improving Vessel Segmentation with Multi-Task Learning and Auxiliary Data Available Only During Model Training
• GRAM-R²: Self-Training Generative Foundation Reward Models for Reward Reasoning
• Clustering-based aggregate value regression
• One Joke to Rule them All? On the (Im)possibility of Generalizing Humor
• See Beyond a Single View: Multi-Attribution Learning Leads to Better Conversion Rate Prediction
• Tensorized Multi-Task Learning for Personalized Modeling of Heterogeneous Individuals with High-Dimensional Data
• Cross-Cancer Knowledge Transfer in WSI-based Prognosis Prediction
• RICO: Two Realistic Benchmarks and an In-Depth Analysis for Incremental Learning in Object Detection
• Vision Generalist Model: A Survey
• Can Multitask Learning Enhance Model Explainability?
• Small transformer architectures for task switching
• Slice or the Whole Pie? Utility Control for AI Models
• Beyond Hard Sharing: Efficient Multi-Task Speech-to-Text Modeling with Supervised Mixture of Experts
• EoH-S: Evolution of Heuristic Set using LLMs for Automated Heuristic Design
• Separating Shared and Domain-Specific LoRAs for Multi-Domain Learning
• Momentum-integrated Multi-task Stock Recommendation with Converge-based Optimization
• Replay in Deep Learning: Current Approaches and Missing Biological Elements
• Weakly Supervised Multimodal Temporal Forgery Localization via Multitask Learning
• A New Semi-supervised Learning Benchmark for Classifying View and Diagnosing Aortic Stenosis from Echocardiograms
• Seeing More with Less: Video Capsule Endoscopy with Multi-Task Learning
• The hierarchical barycenter: conditional probability simulation with structured and unobserved covariates
• Formal Bayesian Transfer Learning via the Total Risk Prior
• Multi-Task Neural Processes
• Boosting Team Modeling through Tempo-Relational Representation Learning
• A Survey on Deep Multi-Task Learning in Connected Autonomous Vehicles
• MTL-UE: Learning to Learn Nothing for Multi-Task Learning
• Rep-MTL: Unleashing the Power of Representation-level Task Saliency for Multi-Task Learning
• Multi-output Deep-Supervised Classifier Chains for Plant Pathology
• Enhancing Stability of Physics-Informed Neural Network Training Through Saddle-Point Reformulation
• A Novel Coded Computing Approach for Distributed Multi-Task Learning
• Integrated Radio Sensing Capabilities for 6G Networks: AI/ML Perspective
• Towards channel foundation models (CFMs): Motivations, methodologies and opportunities
• DEMONSTRATE: Zero-shot Language to Robotic Control via Multi-task Demonstration Learning
• From Time-series Generation, Model Selection to Transfer Learning: A Comparative Review of Pixel-wise Approaches for Large-scale Crop Mapping
• Distribution Matching for Heterogeneous Multi-Task Learning: a Large-scale Face Study
• Robust-Multi-Task Gradient Boosting
• A conditional multi-label model to improve prediction of a rare outcome: An illustration predicting autism diagnosis
• CanadaFireSat: Toward high-resolution wildfire forecasting with multiple modalities
• SAMO: A Lightweight Sharpness-Aware Approach for Multi-Task Optimization with Joint Global-Local Perturbation
• Efficient and Scalable Estimation of Distributional Treatment Effects with Multi-Task Neural Networks
• Multi-Task Multi-Agent Reinforcement Learning via Skill Graphs
• Efficient Multi-Task Reinforcement Learning with Cross-Task Policy Guidance
• Physics-informed neural networks with adaptive loss weighting algorithm for solving partial differential equations
• XiYan-SQL: A Novel Multi-Generator Framework For Text-to-SQL
• Enhancing Learning Path Recommendation via Multi-task Learning
• From Query to Explanation: Uni-RAG for Multi-Modal Retrieval-Augmented Learning in STEM
• TD-MPC-Opt: Distilling Model-Based Multi-Task Reinforcement Learning Agents
• Bayesian Joint Model of Multi-Sensor and Failure Event Data for Multi-Mode Failure Prediction
• Pre-training Time Series Models with Stock Data Customization
• RealDriveSim: A Realistic Multi-Modal Multi-Task Synthetic Dataset for Autonomous Driving
• Bandit Algorithms for Precision Medicine
• MoORE: SVD-based Model MoE-ization for Conflict- and Oblivion-Resistant Multi-Task Adaptation
• CARoL: Context-aware Adaptation for Robot Learning
• Collaborative Multi-LoRA Experts with Achievement-based Multi-Tasks Loss for Unified Multimodal Information Extraction
• StatsMerging: Statistics-Guided Model Merging via Task-Specific Teacher Distillation
• NN-Based Joint Mitigation of IQ Imbalance and PA Nonlinearity With Multiple States
• Multi-task Learning with Active Learning for Arabic Offensive Speech Detection
• Enhancing Customer Service Chatbots with Context-Aware NLU through Selective Attention and Multi-task Learning
• Improving Small-Area Estimates of Public Opinion by Calibrating to Known Population Quantities
• Towards Unified Modeling in Federated Multi-Task Learning via Subspace Decoupling
• Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Data
• FastCAR: Fast Classification And Regression for Task Consolidation in Multi-Task Learning to Model a Continuous Property Variable of Detected Object Class
• PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregressive Reward Model
• Towards Robust Overlapping Speech Detection: A Speaker-Aware Progressive Approach Using WavLM
• DeepChest: Dynamic Gradient-Free Task Weighting for Effective Multi-Task Learning in Chest X-ray Classification
• MoRE: A Mixture of Low-Rank Experts for Adaptive Multi-Task Learning
• Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging
• Multi-Mode Process Control Using Multi-Task Inverse Reinforcement Learning
• GeoERM: Geometry-Aware Multi-Task Representation Learning on Riemannian Manifolds
• ThanoRA: Task Heterogeneity-Aware Multi-Task Low-Rank Adaptation
• Knowledge Grafting of Large Language Models
• Time-o1: Time-Series Forecasting Needs Transformed Label Alignment
• Point, Detect, Count: Multi-Task Medical Image Understanding with Instruction-Tuned Vision-Language Models
• Deep Learning-Based Active Trim Panels for Enhanced Aircraft Interior Noise Control
• Enhancing Epidemic Forecasting: Evaluating the Role of Mobility Data and Graph Convolutional Networks
• Transfer Learning from Visual Speech Recognition to Mouthing Recognition in German Sign Language
• Bridging Speech Emotion Recognition and Personality: Dataset and Temporal Interaction Condition Network
• XDementNET: An Explainable Attention Based Deep Convolutional Network to Detect Alzheimer Progression from MRI data
• Learning Cross-Task Features With Mamba for Remote Sensing Image Multitask Prediction
• Towards Better Evaluation for Generated Patent Claims
• Towards a safe and efficient clinical implementation of machine learning in radiation oncology by exploring model interpretability, explainability and data-model dependency
• End-to-End Multi-Task Policy Learning from NMPC for Quadruped Locomotion
• CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging
• RiM: Record, Improve and Maintain Physical Well-being using Federated Learning
• Boosting Statistic Learning with Synthetic Data from Pretrained Large Models
• One Model, Many Skills: Parameter-Efficient Fine-Tuning for Multitask Code Analysis
• PFAdapter: Hierarchical LoRA Decomposition for Personalized Federated MLLMs
• DynoSys: A Dynamic Systems Framework for Multimodal Integration of Genetic, Environmental, and Neurobiological Signals
• Multi-Message Secure Aggregation with Demand Privacy
• Parameter-efficient Quantum Multi-task Learning
• DIMT25@ICDAR2025: HW-TSC's End-to-End Document Image Machine Translation System Leveraging Large Vision-Language Model
• Towards a fast and robust deep hedging approach
• MTSGL: Multi-Task Structure Guided Learning for Robust and Interpretable SAR Aircraft Recognition
• Efficient Multi-Task Modeling through Automated Fusion of Trained Models
• Multi-task Federated Learning with Encoder-Decoder Structure: Enabling Collaborative Learning Across Different Tasks
• Multi-task neural networks by learned contextual inputs
• Two-Stage nnU-Net for Automatic Multi-class Bi-Atrial Segmentation from LGE-MRIs

Related

• A Survey on Multi-Task Learning
• Multi-Task Learning with Deep Neural Networks: A Survey
• A Brief Review of Deep Multi-task Learning and Auxiliary Task Learning
• Design Perspectives of Multitask Deep Learning Models and Applications
• Towards Principled Task Grouping for Multi-Task Learning