2018/12/30 by H. Sebastian Seung, Seung, H. Sebastian · 1 citation
Computer Science · Engineering · Neuroscience · #Advanced Memory and Neural Computing #FOS: Biological sciences #FOS: Computer and information sciences #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Neural dynamics and brain function #Neurons and Cognition (q-bio.NC)
paper · pdf · doi:10.48550/arxiv.1812.11581
openalex publication_date 2018/12/30 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
This paper introduces a rate-based nonlinear neural network in which\nexcitatory (E) neurons receive feedforward excitation from sensory (S) neurons,\nand inhibit each other through disynaptic pathways mediated by inhibitory (I)\ninterneurons. Correlation-based plasticity of disynaptic inhibition serves to\nincompletely decorrelate E neuron activity, pushing the E neurons to learn\ndistinct sensory features. The plasticity equations additionally contain\n"extra" terms fostering competition between excitatory synapses converging onto\nthe same postsynaptic neuron and inhibitory synapses diverging from the same\npresynaptic neuron. The parameters of competition between S\→E connections\ncan be adjusted to make learned features look more like "parts" or "wholes."\nThe parameters of competition between I-E connections can be adjusted to set\nthe typical decorrelatedness and sparsity of E neuron activity. Numerical\nsimulations of unsupervised learning show that relatively few I neurons can be\nsufficient for achieving good decorrelation, and increasing the number of I\nneurons makes decorrelation more complete. Excitatory and inhibitory inputs to\nactive E neurons are approximately balanced as a result of learning.\n