2016/11/10 by Jeffrey Mark Siskind, Barak A. Pearlmutter, Siskind, Jeffrey Mark +1
Computer Science · Engineering · #Advanced Control Systems Optimization #FOS: Computer and information sciences #Mathematical Software (cs.MS) #Numerical Methods and Algorithms #Parallel Computing and Optimization Techniques #Programming Languages (cs.PL) #cs.MS #cs.PL
paper · pdf · doi:10.48550/arxiv.1611.03416
Extended abstract presented at the AD 2016 Conference, Sep 2016, Oxford UK
arxiv created 2016/11/10 · openalex publication_date 2016/11/10 · arxiv updated 2016/11/11 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
We show that Automatic Differentiation (AD) operators can be provided in a dynamic language without sacrificing numeric performance. To achieve this, general forward and reverse AD functions are added to a simple high-level dynamic language, and support for them is included in an aggressive optimizing compiler. Novel technical mechanisms are discussed, which have the ability to migrate the AD transformations from run-time to compile-time. The resulting system, although only a research prototype, exhibits startlingly good performance. In fact, despite the potential inefficiencies entailed by support of a functional-programming language and a first-class AD operator, performance is competitive with the fastest available preprocessor-based Fortran AD systems. On benchmarks involving nested use of the AD operators, it can even dramatically exceed their performance.