A Common Interface for Automatic Differentiation

2025/05/08 by Guillaume Dalle, Adrian Hill, Dalle, Guillaume +1 · 2 voices · 2 citations
Computer Science · Physics and Astronomy · #Model Reduction and Neural Networks #Numerical Methods and Algorithms #Parallel Computing and Optimization Techniques #cs.LG #cs.MS #math.NA

paper · pdf · doi:10.48550/arxiv.2505.05542

openalex publication_date 2025/05/08 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28

Abstract

For scientific machine learning tasks with a lot of custom code, picking the right Automatic Differentiation (AD) system matters. Our Julia package DifferentiationInterface.jl provides a common frontend to a dozen AD backends, unlocking easy comparison and modular development. In particular, its built-in preparation mechanism leverages the strengths of each backend by amortizing one-time computations. This is key to enabling sophisticated features like sparsity handling without putting additional burdens on the user.

Citations

• Revisiting Sparse Matrix Coloring and Bicoloring
• Sparser, Better, Faster, Stronger: Sparsity Detection for Efficient Automatic Differentiation
• PyTorch: An Imperative Style, High-Performance Deep Learning Library
• A Differentiable Programming System to Bridge Machine Learning and Scientific Computing
• Don't Unroll Adjoint: Differentiating SSA-Form Programs
• Automatic differentiation in machine learning: a survey
• Julia: A Fresh Approach to Numerical Computing
• Julia: A Fresh Approach to Numerical Computing

Cited by

• Efficient optimisation of multi-parameter quantum control protocols for strongly-coupled systems
• Sparser, Better, Faster, Stronger: Sparsity Detection for Efficient Automatic Differentiation
• Algorithmic differentiation for plane-wave DFT: materials design, error control and learning model parameters

Discussions

• How to make #autodiff user-friendly? What lies beyond the safety of Python-world? Why does it matter for scientific machine learning? All this, and more, in our latest preprint with @adrhill.bsky.soci [bsky, 51 points, 1 comments]
• Guillaume Dalle, Adrian Hill: A Common Interface for Automatic Differentiation https://arxiv.org/abs/2505.05542 https://arxiv.org/pdf/2505.05542 https://arxiv.org/html/2505.05542 [bsky, 0 points, 1 comments]

Related

• A Common Interface for Automatic Differentiation
• AbstractDifferentiation.jl: Backend-Agnostic Differentiable Programming in Julia
• DaCe AD: Unifying High-Performance Automatic Differentiation for Machine Learning and Scientific Computing
• A Differentiable Programming System to Bridge Machine Learning and Scientific Computing
• Forward-Mode Automatic Differentiation in Julia