Multi-Activation Hidden Units for Neural Networks with Random Weights

2020/09/06 by Ajay M. Patrikar, Patrikar, Ajay M.
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and ELM #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.2009.08932

openalex publication_date 2020/09/06 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Single layer feedforward networks with random weights are successful in a variety of classification and regression problems. These networks are known for their non-iterative and fast training algorithms. A major drawback of these networks is that they require a large number of hidden units. In this paper, we propose the use of multi-activation hidden units. Such units increase the number of tunable parameters and enable formation of complex decision surfaces, without increasing the number of hidden units. We experimentally show that multi-activation hidden units can be used either to improve the classification accuracy, or to reduce computations.

Citations

• Functional-link net computing: theory, system architecture, and functionalities
• Gradient-based learning applied to document recognition
• A survey on modern trainable activation functions

Related

• Efficient Design of Neural Networks with Random Weights
• Half-Layered Neural Networks
• Techniques for Learning Binary Stochastic Feedforward Neural Networks
• Empirical Explorations in Training Networks with Discrete Activations
• A Method of Generating Random Weights and Biases in Feedforward Neural Networks with Random Hidden Nodes