2014/07/22 by Ila Fiete, David J. Schwab, Fiete, Ila +3
Biochemistry, Genetics and Molecular Biology · Mathematics · #Dynamical Systems (math.DS) #FOS: Biological sciences #FOS: Mathematics #Neurons and Cognition (q-bio.NC) #math.DS #q-bio.NC
paper · pdf · doi:10.48550/arxiv.1407.6029
extended abstract, 4 pages, 2 figures
arxiv created 2014/07/22 · arxiv updated 2014/07/24
A Hopfield network is an auto-associative, distributive model of neural memory storage and retrieval. A form of error-correcting code, the Hopfield network can learn a set of patterns as stable points of the network dynamic, and retrieve them from noisy inputs -- thus Hopfield networks are their own decoders. Unlike in coding theory, where the information rate of a good code (in the Shannon sense) is finite but the cost of decoding does not play a role in the rate, the information rate of Hopfield networks trained with state-of-the-art learning algorithms is of the order log(n)/n, a quantity that tends to zero asymptotically with n, the number of neurons in the network. For specially constructed networks, the best information rate currently achieved is of order 1/√(n). In this work, we design simple binary Hopfield networks that have asymptotically vanishing error rates at an information rate of 1/log(n). These networks can be added as the decoders of any neural code with noisy neurons. As an example, we apply our network to a binary neural decoder of the grid cell code to attain information rate 1/log(n).