Learning representations by forward-propagating errors

2023/08/17 by Ryoungwoo Jang, Jang, Ryoungwoo
Computer Science · Engineering · #Advanced Numerical Analysis Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Polynomial and algebraic computation

paper · pdf · doi:10.48550/arxiv.2308.09728

openalex publication_date 2023/08/17 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Back-propagation (BP) is widely used learning algorithm for neural network optimization. However, BP requires enormous computation cost and is too slow to train in central processing unit (CPU). Therefore current neural network optimizaiton is performed in graphical processing unit (GPU) with compute unified device architecture (CUDA) programming. In this paper, we propose a light, fast learning algorithm on CPU that is fast as CUDA acceleration on GPU. This algorithm is based on forward-propagating method, using concept of dual number in algebraic geometry.

Related

• Large Scale Artificial Neural Network Training Using Multi-GPUs
• Neural GPUs Learn Algorithms
• Optimization Techniques to Improve Inference Performance of a Forward Propagating Neural Network on an FPGA
• FF-INT8: Efficient Forward-Forward DNN Training on Edge Devices with INT8 Precision
• On-FPGA Training with Ultra Memory Reduction: A Low-Precision Tensor Method