PCA-based Multi Task Learning: a Random Matrix Approach

2021/11/01 by Malik Tiomoko, Romain Couillet, Tiomoko, Malik +3 · 1 citation
Computer Science · #Face and Expression Recognition #Domain Adaptation and Few-Shot Learning #Machine Learning and ELM

paper · pdf · doi:10.48550/arxiv.2111.00924

Abstract

The article proposes and theoretically analyses a computationally efficient multi-task learning (MTL) extension of popular principal component analysis (PCA)-based supervised learning schemes \citebarshan2011supervised,bair2006prediction. The analysis reveals that (i) by default learning may dramatically fail by suffering from negative transfer, but that (ii) simple counter-measures on data labels avert negative transfer and necessarily result in improved performances. Supporting experiments on synthetic and real data benchmarks show that the proposed method achieves comparable performance with state-of-the-art MTL methods but at a significantly reduced computational cost.

Citations

• On the distribution of the largest eigenvalue in principal components analysis
• Unsupervised Domain Adaptation with Residual Transfer Networks
• An Overview of Multi-Task Learning in Deep Neural Networks

Cited by

• Incorporating priors in learning: a random matrix study under a teacher-student framework

Related

• Multi-task learning on the edge: cost-efficiency and theoretical optimality
• Multi-Task Learning on Networks
• TL-PCA: Transfer Learning of Principal Component Analysis
• A Large Dimensional Analysis of Multi-task Semi-Supervised Learning
• Supervised PCA: A Multiobjective Approach