Matrix Calculus (for Machine Learning and Beyond)
2025/01/07 by Paige Bright, Alan Edelman, Bright, Paige +3 · 13 voices · 2 citations
Computer Science · #Neural Networks and Applications #cs.LG #math.HO #math.NA #stat.ML
paper · pdf · doi:10.48550/arxiv.2501.14787
openalex publication_date 2025/01/07 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
Abstract
This course, intended for undergraduates familiar with elementary calculus and linear algebra, introduces the extension of differential calculus to functions on more general vector spaces, such as functions that take as input a matrix and return a matrix inverse or factorization, derivatives of ODE solutions, and even stochastic derivatives of random functions. It emphasizes practical computational applications, such as large-scale optimization and machine learning, where derivatives must be re-imagined in order to be propagated through complicated calculations. The class also discusses efficiency concerns leading to "adjoint" or "reverse-mode" differentiation (a.k.a. "backpropagation"), and gives a gentle introduction to modern automatic differentiation (AD) techniques.
Cited by
Discussions
- Matrix Calculus (For Machine Learning and Beyond) [hn, 183 points, 30 comments]
- Looks good arxiv.org/abs/2501.14787 [bsky, 9 points, 1 comments]
- Paige Bright, Alan Edelman, Steven G. Johnson: Matrix Calculus (for Machine Learning and Beyond) https://arxiv.org/abs/2501.14787 https://arxiv.org/pdf/2501.14787 https://arxiv.org/html/2501.14787 [bsky, 2 points, 1 comments]
- Matrix Calculus (For Machine Learning and Beyond) [hn, 1 points, 0 comments]
- Matrix Calculus (For Machine Learning and Beyond) https://arxiv.org/abs/2501.14787 [bsky, 1 points, 0 comments]
- Matrix Calculus (For Machine Learning and Beyond) https://arxiv.org/abs/2501.14787 https://news.ycombinator.com/item?id=43518220 [bsky, 0 points, 0 comments]
- Matrix Calculus (For Machine Learning and Beyond) https://arxiv.org/abs/2501.14787 [bsky, 0 points, 0 comments]
- Matrix Calculus (For Machine Learning and Beyond) https://arxiv.org/abs/2501.14787 [comments] [119 points] [bsky, 0 points, 0 comments]
- Matrix Calculus (For Machine Learning and Beyond) #ai #machinelearning #news [bsky, 0 points, 0 comments]
- Matrix Calculus (For Machine Learning and Beyond) https://arxiv.org/abs/2501.14787 (https://news.ycombinator.com/item?id=43518220) [bsky, 0 points, 0 comments]
- Matrix Calculus (for Machine Learning and Beyond) arxiv.org/pdf/2501.14787 [bsky, 0 points, 0 comments]
- Матричный исчисление (для машинного обучения и не только) #ai #machinelearning #news [bsky, 0 points, 0 comments]
- Matrix Calculus (For Machine Learning and Beyond) [bsky, 0 points, 0 comments]
Related