2024/02/23 by Bruno Gavranović, Paul Lessard, Gavranović, Bruno +9 · 2 voices · 5 citations
Computer Science · Engineering · Mathematics · #Artificial Intelligence (cs.AI) #Category Theory (math.CT) #Data Visualization and Analytics #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Psychiatry, Mental Health, Neuroscience #Rings and Algebras (math.RA) #Urban Design and Spatial Analysis #cs.AI #cs.LG #math.CT #math.RA #stat.ML
paper · pdf · doi:10.48550/arxiv.2402.15332
openalex publication_date 2024/02/23 · arxiv published 2024/02/23 · openalex created_date 2024/02/27 · arxiv updated 2024/06/06 · openalex updated_date 2026/07/28
We present our position on the elusive quest for a general-purpose framework for specifying and studying deep learning architectures. Our opinion is that the key attempts made so far lack a coherent bridge between specifying constraints which models must satisfy and specifying their implementations. Focusing on building a such a bridge, we propose to apply category theory -- precisely, the universal algebra of monads valued in a 2-category of parametric maps -- as a single theory elegantly subsuming both of these flavours of neural network design. To defend our position, we show how this theory recovers constraints induced by geometric deep learning, as well as implementations of many architectures drawn from the diverse landscape of neural networks, such as RNNs. We also illustrate how the theory naturally encodes many standard constructs in computer science and automata theory.