Deep Multi-task Representation Learning: A Tensor Factorisation Approach

2016/05/20 by Yongxin Yang, Yang, Yongxin, Timothy M. Hospedales +2 · 19 citations
Computer Science · Mathematics · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Tensor decomposition and applications #cs.LG

paper · pdf · doi:10.48550/arxiv.1605.06391

9 pages, Accepted to ICLR 2017 Conference Track. This is a conference version of the paper. For the multi-domain learning part (not in this version), please refer to https://arxiv.org/pdf/1605.06391v1.pdf

openalex publication_date 2016/05/20 · arxiv created 2017/02/16 · arxiv updated 2017/02/20 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Most contemporary multi-task learning methods assume linear models. This setting is considered shallow in the era of deep learning. In this paper, we present a new deep multi-task representation learning framework that learns cross-task sharing structure at every layer in a deep network. Our approach is based on generalising the matrix factorisation techniques explicitly or implicitly used by many conventional MTL algorithms to tensor factorisation, to realise automatic learning of end-to-end knowledge sharing in deep networks. This is in contrast to existing deep learning approaches that need a user-defined multi-task sharing strategy. Our approach applies to both homogeneous and heterogeneous MTL. Experiments demonstrate the efficacy of our deep multi-task representation learning in terms of both higher accuracy and fewer design choices.

Cited by

• Cross-Learning from Scarce Data via Multi-Task Constrained Optimization
• Generalist++: A Meta-learning Framework for Mitigating Trade-off in Adversarial Training
• Tensor Regression Networks
• Tensor Contraction Layers for Parsimonious Deep Nets
• Provable Adaptation across Multiway Domains via Representation Learning
• Multi-Task Generative Adversarial Nets with Shared Memory for Cross-Domain Coordination Control
• Auxiliary Learning for Deep Multi-task Learning
• Modeling the Sequential Dependence among Audience Multi-step Conversions with Multi-task Learning in Targeted Display Advertising
• Deep Multi-Task Augmented Feature Learning via Hierarchical Graph Neural Network
• Compositional Models: Multi-Task Learning and Knowledge Transfer with Modular Networks
• Deep Multi-Task Learning via Generalized Tensor Trace Norm
• Evolutionary Architecture Search For Deep Multitask Networks
• Multi-Task Zipping via Layer-wise Neuron Sharing
• A Simple General Approach to Balance Task Difficulty in Multi-Task Learning
• From Persistent Homology to Reinforcement Learning with Applications for Retail Banking
• Adaptive Activation Network and Functional Regularization for Efficient and Flexible Deep Multi-Task Learning
• Deep Stacking Networks for Low-Resource Chinese Word Segmentation with Transfer Learning
• Trace Norm Regularised Deep Multi-Task Learning
• Convolutional Neural Networks with Transformed Input based on Robust Tensor Network Decomposition

Related

• Factorized Tensor Networks for Multi-Task and Multi-Domain Learning
• Multi-Task Learning with Deep Neural Networks: A Survey
• Trace Norm Regularised Deep Multi-Task Learning
• Learning Multiple Tasks with Multilinear Relationship Networks
• Latent Multi-task Architecture Learning