On the relationship between disentanglement and multi-task learning

2021/10/07 by Łukasz Maziarka, Aleksandra Nowak, Maziarka, Łukasz +5 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Digital Media Forensic Detection #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)

paper · pdf · doi:10.48550/arxiv.2110.03498

openalex publication_date 2021/10/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

One of the main arguments behind studying disentangled representations is the assumption that they can be easily reused in different tasks. At the same time finding a joint, adaptable representation of data is one of the key challenges in the multi-task learning setting. In this paper, we take a closer look at the relationship between disentanglement and multi-task learning based on hard parameter sharing. We perform a thorough empirical study of the representations obtained by neural networks trained on automatically generated supervised tasks. Using a set of standard metrics we show that disentanglement appears naturally during the process of multi-task neural network training.

Cited by

• Multi-task Learning for Heterogeneous Multi-source Block-Wise Missing Data

Related

• Leveraging sparse and shared feature activations for disentangled representation learning
• Synergies between Disentanglement and Sparsity: Generalization and Identifiability in Multi-Task Learning
• Revisiting Disentanglement in Downstream Tasks: A Study on Its Necessity for Abstract Visual Reasoning
• Disentangled Representations in Neural Models
• Theory and Evaluation Metrics for Learning Disentangled Representations