About Learning in Recurrent Bistable Gradient Networks

2016/08/29 by J.R. Fischer, Fischer, J., S. Lackner +1
Computer Science · #FOS: Computer and information sciences #Machine Learning and ELM #Neural Networks Stability and Synchronization #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)

paper · pdf · doi:10.48550/arxiv.1608.08265

openalex publication_date 2016/08/29 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Recurrent Bistable Gradient Networks are attractor based neural networks characterized by bistable dynamics of each single neuron. Coupled together using linear interaction determined by the interconnection weights, these networks do not suffer from spurious states or very limited capacity anymore. Vladimir Chinarov and Michael Menzinger, who invented these networks, trained them using Hebb's learning rule. We show, that this way of computing the weights leads to unwanted behaviour and limitations of the networks capabilities. Furthermore we evince, that using the first order of Hintons Contrastive Divergence algorithm leads to a quite promising recurrent neural network. These findings are tested by learning images of the MNIST database for handwritten numbers.

Related

• Bistable Gradient Networks in the Thermodynamic Limit
• Rethinking the Relationship between Recurrent and Non-Recurrent Neural Networks: A Study in Sparsity
• Recurrent neural networks: vanishing and exploding gradients are not the end of the story
• Brain-like combination of feedforward and recurrent network components achieves prototype extraction and robust pattern recognition
• On the difficulty of training Recurrent Neural Networks