Memory Capacity of a Random Neural Network

2012/11/14 by Stowe, Matt
#FOS: Computer and information sciences #Neural and Evolutionary Computing (cs.NE)

paper · doi:10.48550/arxiv.1211.3451

Abstract

This paper considers the problem of information capacity of a random neural network. The network is represented by matrices that are square and symmetrical. The matrices have a weight which determines the highest and lowest possible value found in the matrix. The examined matrices are randomly generated and analyzed by a computer program. We find the surprising result that the capacity of the network is a maximum for the binary random neural network and it does not change as the number of quantization levels associated with the weights increases.

Related

• Memory Capacity of Nonlinear Recurrent Networks: Is it Informative?
• Memory Capacity of Recurrent Neural Networks with Matrix Representation
• On the capacity of neural networks
• A Capacity Scaling Law for Artificial Neural Networks
• Three unfinished works on the optimal storage capacity of networks