Felix Dangel

1. BackPACK: Packing more into backprop
   2019/12/23 by Felix Dangel, Dangel, Felix, Frederik Künstner +3 · 4 citations
   Computer Science · Engineering · Mathematics · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #Tensor decomposition and applications
2. Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
   2024/02/05 by Lin Wu, Felix Dangel, Lin, Wu +9 · 3 citations
   Mathematics · #Advanced Optimization Algorithms Research #FOS: Computer and information sciences #FOS: Mathematics #Iterative Methods for Nonlinear Equations #Machine Learning (cs.LG) #Numerical methods in inverse problems #Optimization and Control (math.OC)
3. What Does It Mean to Be a Transformer? Insights from a Theoretical Hessian Analysis
   2024/10/14 by Weronika Ormaniec, Ormaniec, Weronika, Felix Dangel +3 · 3 citations
   Engineering · Materials Science · #Control and Stability of Dynamical Systems #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Magnetic Properties and Applications
4. On the Disconnect Between Theory and Practice of Neural Networks: Limits of the NTK Perspective
   2023/09/29 by Jonathan Wenger, Wenger, Jonathan, Felix Dangel +3 · 2 citations
   Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks #Neural Networks and Applications #Stochastic Gradient Optimization Techniques
5. Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization
   2025/05/17 by Andrés Guzmán-Cordero, Felix Dangel, Guzmán-Cordero, Andrés +5 · 6 citations
   Computer Science · Physics and Astronomy · Materials Science · #Stochastic Gradient Optimization Techniques #Model Reduction and Neural Networks #Machine Learning in Materials Science
6. Fishers for Free? Approximating the Fisher Information Matrix by Recycling the Squared Gradient Accumulator
   2025/07/24 by Yuxin Li, Felix Dangel, Li, YuXin +5 · 4 citations
   Computer Science · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Metaheuristic Optimization Algorithms Research #Stochastic Gradient Optimization Techniques
7. Structured Inverse-Free Natural Gradient: Memory-Efficient & Numerically-Stable KFAC
   2023/12/09 by Lin Wu, Lin, Wu, Felix Dangel +11 · 1 citation
   Computer Science · Earth and Planetary Sciences · Physics and Astronomy · #FOS: Computer and information sciences #Geophysical and Geoelectrical Methods #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Matrix Theory and Algorithms #Model Reduction and Neural Networks