2022/07/17 by Naichen Shi, Shi, Naichen, Raed Al Kontar +1 · 3 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #FOS: Computer and information sciences #FOS: Mathematics #Image Retrieval and Classification Techniques #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistics Theory (math.ST) #Text and Document Classification Technologies
paper · pdf · doi:10.48550/arxiv.2207.08041
openalex publication_date 2022/07/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
In this paper, we tackle a significant challenge in PCA: heterogeneity. When data are collected from different sources with heterogeneous trends while still sharing some congruency, it is critical to extract shared knowledge while retaining the unique features of each source. To this end, we propose personalized PCA (PerPCA), which uses mutually orthogonal global and local principal components to encode both unique and shared features. We show that, under mild conditions, both unique and shared features can be identified and recovered by a constrained optimization problem, even if the covariance matrices are immensely different. Also, we design a fully federated algorithm inspired by distributed Stiefel gradient descent to solve the problem. The algorithm introduces a new group of operations called generalized retractions to handle orthogonality constraints, and only requires global PCs to be shared across sources. We prove the linear convergence of the algorithm under suitable assumptions. Comprehensive numerical experiments highlight PerPCA's superior performance in feature extraction and prediction from heterogeneous datasets. As a systematic approach to decouple shared and unique features from heterogeneous datasets, PerPCA finds applications in several tasks, including video segmentation, topic extraction, and feature clustering.