2022/06/14 by Felipe Farias, Farias, Felipe Costa, Teresa B. Ludermir +3
Computer Science · Neuroscience · #Brain Tumor Detection and Classification #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Stochastic Gradient Optimization Techniques
paper · pdf · doi:10.48550/arxiv.2206.08369
openalex publication_date 2022/06/14 · openalex created_date 2022/06/22 · openalex updated_date 2026/07/28
The definition of a Neural Network architecture is one of the most critical and challenging tasks to perform. In this paper, we propose ParallelMLPs. ParallelMLPs is a procedure to enable the training of several independent Multilayer Perceptron Neural Networks with a different number of neurons and activation functions in parallel by exploring the principle of locality and parallelization capabilities of modern CPUs and GPUs. The core idea of this technique is to use a Modified Matrix Multiplication that replaces an ordinal matrix multiplication by two simple matrix operations that allow separate and independent paths for gradient flowing, which can be used in other scenarios. We have assessed our algorithm in simulated datasets varying the number of samples, features and batches using 10,000 different models. We achieved a training speedup from 1 to 4 orders of magnitude if compared to the sequential approach.