Sigmoid generators for neural computing using piecewise approximations

1996/01/01 by Ming Zhang, S. Vassiliadis, J.G. Delgado-Frias · 7 citations
Computer Science · Physics and Astronomy · #Neural Networks and Applications #Model Reduction and Neural Networks #Numerical Methods and Algorithms

paper · doi:10.1109/12.537127

Abstract

A piecewise second order approximation scheme is proposed for computing the sigmoid function. The scheme provides high performance with low implementation cost; thus, it is suitable for hardwired cost effective neural emulators. It is shown that an implementation of the sigmoid generator outperforms, in both precision and speed, existing schemes using a bit serial pipelined implementation. The proposed generator requires one multiplication, no look-up table and no addition. It has been estimated that the sigmoid output is generated with a maximum computation delay of 21 bit serial machine cycles representing a speedup of 1.57 to 2.23 over other proposals.

Cited by

• Efficient VLSI Implementation of Neural Networks With Hyperbolic Tangent Activation Function
• On Training Efficiency and Computational Costs of a Feed Forward Neural Network: A Review. [europepmc]
• FPGA-Based Implementation of Stochastic Configuration Networks for Regression Prediction. [europepmc]
• FPGA-Based High-Performance Phonocardiography System for Extraction of Cardiac Sound Components Using Inverse Delayed Neuron Model. [europepmc]
• Hardware-Based Architecture for DNN Wireless Communication Models. [europepmc]
• QuantLaneNet: A 640-FPS and 34-GOPS/W FPGA-Based CNN Accelerator for Lane Detection. [europepmc]
• Efficient Neural Networks on the Edge with FPGAs by Optimizing an Adaptive Activation Function. [europepmc]

Related

• About rectified sigmoid function for enhancing the accuracy of Physics-Informed Neural Networks
• Approximation by Exponential Type Neural Network Operators
• Efficient VLSI Implementation of Neural Networks With Hyperbolic Tangent Activation Function
• Formal Analysis of the Sigmoid Function and Formal Proof of the Universal Approximation Theorem
• Constructing Multilayer Perceptrons as Piecewise Low-Order Polynomial Approximators: A Signal Processing Approach