Deep Feedback Models

2025/09/19 by David Calhas, Calhas, David, Arlindo L. Oliveira +1
Computer Science · Physics and Astronomy · #Adversarial Robustness in Machine Learning #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Model Reduction and Neural Networks

paper · pdf · doi:10.48550/arxiv.2509.15905

openalex publication_date 2025/09/19 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

Deep Feedback Models (DFMs) are a new class of stateful neural networks that combine bottom up input with high level representations over time. This feedback mechanism introduces dynamics into otherwise static architectures, enabling DFMs to iteratively refine their internal state and mimic aspects of biological decision making. We model this process as a differential equation solved through a recurrent neural network, stabilized via exponential decay to ensure convergence. To evaluate their effectiveness, we measure DFMs under two key conditions: robustness to noise and generalization with limited data. In both object recognition and segmentation tasks, DFMs consistently outperform their feedforward counterparts, particularly in low data or high noise regimes. In addition, DFMs translate to medical imaging settings, while being robust against various types of noise corruption. These findings highlight the importance of feedback in achieving stable, robust, and generalizable learning. Code is available at https://github.com/DCalhas/deep_feedback_models.

Citations

• MedMNIST-C: Comprehensive benchmark and improved classifier robustness by simulating realistic image corruptions
• Computing a human-like reaction time metric from stable recurrent vision models
• Feasibility-based Fixed Point Networks
• Stable and expressive recurrent vision models
• Deep Equilibrium Models
• Recurrent U-Net for Resource-Constrained Segmentation
• Riemannian Adaptive Optimization Methods
• Reviving and Improving Recurrent Back-Propagation
• Equivalence of Equilibrium Propagation and Recurrent Backpropagation
• Cyclical Learning Rates for Training Neural Networks
• Microsoft COCO: Common Objects in Context

Related

• Deep Recurrence for Dynamical Segmentation Models
• On the role of feedback in visual processing: a predictive coding perspective
• Deep Neural Networks
• Predify: Augmenting deep neural networks with brain-inspired predictive coding dynamics
• Recurrent Feedback Improves Feedforward Representations in Deep Neural Networks