Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics

2021/06/04 by Bhavin Choksi, Milad Mozafari, Choksi, Bhavin +10 · 25 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · Neuroscience · #Adversarial Robustness in Machine Learning #Artificial intelligence #Artificial neural network #Cell Image Analysis Techniques #Coding (social sciences) #Computational neuroscience #Computer Vision and Pattern Recognition (cs.CV) #Computer science #Convolutional neural network #Deep learning #Deep neural networks #FOS: Biological sciences #FOS: Computer and information sciences #Feed forward #Generative grammar #Machine learning #Neural coding #Neural dynamics and brain function #Neurons and Cognition (q-bio.NC) #Pattern recognition (psychology) #Predictive coding #Recurrent neural network #Robustness (evolution) #cs.CV #q-bio.NC

paper · pdf · doi:10.48550/arxiv.2106.02749

published in arXiv (Cornell University) (Cornell University) · Paper presented at NeurIPS 2021

openalex publication_date 2021/06/04 · arxiv created 2021/11/04 · arxiv updated 2021/11/05 · openalex created_date 2025/10/10 · openalex updated_date 2026/08/05

Abstract

Deep neural networks excel at image classification, but their performance is far less robust to input perturbations than human perception. In this work we explore whether this shortcoming may be partly addressed by incorporating brain-inspired recurrent dynamics in deep convolutional networks. We take inspiration from a popular framework in neuroscience: 'predictive coding'. At each layer of the hierarchical model, generative feedback 'predicts' (i.e., reconstructs) the pattern of activity in the previous layer. The reconstruction errors are used to iteratively update the network's representations across timesteps, and to optimize the network's feedback weights over the natural image dataset-a form of unsupervised training. We show that implementing this strategy into two popular networks, VGG16 and EfficientNetB0, improves their robustness against various corruptions and adversarial attacks. We hypothesize that other feedforward networks could similarly benefit from the proposed framework. To promote research in this direction, we provide an open-sourced PyTorch-based package called Predify, which can be used to implement and investigate the impacts of the predictive coding dynamics in any convolutional neural network.

Citations

• Intriguing properties of neural networks
• Very Deep Convolutional Networks for Large-Scale Image Recognition
• Explaining and Harnessing Adversarial Examples
• ImageNet classification with deep convolutional neural networks
• Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
• Densely Connected Convolutional Networks
• APE-GAN: Adversarial Perturbation Elimination with GAN
• The Robust Manifold Defense: Adversarial Training using Generative Models
• Deep Predictive Coding Network for Object Recognition
• Adversarially Robust Generalization Requires More Data
• MagNet: a Two-Pronged Defense against Adversarial Examples
• Improving the robustness of ImageNet classifiers using elements of human visual cognition
• PixelDefend: Leveraging Generative Models to Understand and Defend against Adversarial Examples
• Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models
• Attacks Meet Interpretability: Attribute-steered Detection of Adversarial Samples
• Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• Stable and expressive recurrent vision models
• Neural Networks with Recurrent Generative Feedback
• Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
• On Assessing ML Model Robustness: A Methodological Framework (Academic Track)
• Foolbox: A Python toolbox to benchmark the robustness of machine learning models

Cited by

• Model-Behavior Alignment under Flexible Evaluation: When the Best-Fitting Model Isn't the Right One
• Sensory robustness through top-down feedback and neural stochasticity in recurrent vision models
• GASPnet: Global Agreement to Synchronize Phases
• Wave propagation phenomena in nonlinear hierarchical neural networks with predictive coding feedback dynamics

Related

• On the role of feedback in visual processing: a predictive coding perspective
• Deep Predictive Coding Network for Object Recognition
• Deep Predictive Coding Network with Local Recurrent Processing for Object Recognition
• Convolutional Neural Generative Coding: Scaling Predictive Coding to Natural Images
• A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks