2017/06/13 by Sebastian Herzog, Herzog, Sebastian, Christian Tetzlaff +3
Computer Science · Engineering · Mathematics · Neuroscience · #Advanced Memory and Neural Computing #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Neural dynamics and brain function #cs.IT #cs.LG #cs.NE #math.IT
paper · pdf · doi:10.48550/arxiv.1706.04265
openalex publication_date 2017/06/13 · arxiv created 2017/06/22 · arxiv updated 2017/06/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
The structure of the majority of modern deep neural networks is characterized by uni- directional feed-forward connectivity across a very large number of layers. By contrast, the architecture of the cortex of vertebrates contains fewer hierarchical levels but many recurrent and feedback connections. Here we show that a small, few-layer artificial neural network that employs feedback will reach top level performance on a standard benchmark task, otherwise only obtained by large feed-forward structures. To achieve this we use feed-forward transfer entropy between neurons to structure feedback connectivity. Transfer entropy can here intuitively be understood as a measure for the relevance of certain pathways in the network, which are then amplified by feedback. Feedback may therefore be key for high network performance in small brain-like architectures.