2021/05/29 by Firas Hamze, Hamze, Firas
Computer Science · Physics and Astronomy · #Distributed #FOS: Computer and information sciences #Graph Theory and Algorithms #Machine Learning (cs.LG) #Matrix Theory and Algorithms #Model Reduction and Neural Networks #Neural Networks and Applications #Neural Networks and Reservoir Computing #Parallel #and Cluster Computing (cs.DC)
paper · pdf · doi:10.48550/arxiv.2106.00003
openalex publication_date 2021/05/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We present a methodology for parallel acceleration of learning in the presence of matrix orthogonality and unitarity constraints of interest in several branches of machine learning. We show how an apparently sequential elementary rotation parametrization can be restructured into blocks of commutative operations using a well-known tool for coloring the edges of complete graphs, in turn widely applied to schedule round-robin (all-against-all) sports tournaments. The resulting decomposition admits an algorithm to compute a fully-parametrized orthogonal matrix from its rotation parameters in O(n) sequential steps and one to compute the gradient of a training loss with respect to its parameters in O(nlog n) steps. We discuss parametric restrictions of interest to generative modeling and present promising performance results with a prototype GPU implementation.