2017/01/18 by Ludvig Ericson, Ericson, Ludvig, Rendani Mbuvha +1
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (stat.ML) #Machine Learning and ELM #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Performance (cs.PF) #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.1701.05130
openalex publication_date 2017/01/18 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Artificial Neural Networks (ANNs) have received increasing attention in recent years with applications that span a wide range of disciplines including vital domains such as medicine, network security and autonomous transportation. However, neural network architectures are becoming increasingly complex and with an increasing need to obtain real-time results from such models, it has become pivotal to use parallelization as a mechanism for speeding up network training and deployment. In this work we propose an implementation of Network Parallel Training through Cannon's Algorithm for matrix multiplication. We show that increasing the number of processes speeds up training until the point where process communication costs become prohibitive; this point varies by network complexity. We also show through empirical efficiency calculations that the speedup obtained is superlinear.