Rudra, Atri
- FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
2022/05/27 by Tri Dao, Dao, Tri, Daniel Y. Fu +7 · 8 voices · 743 citations
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification #cs.LG
- Hungry Hungry Hippos: Towards Language Modeling with State Space Models
2022/12/28 by Daniel Y. Fu, Fu, Daniel Y., Tri Dao +9 · 3 voices · 55 citations
Computer Science · #Topic Modeling #cs.CL #cs.LG
- HiPPO: Recurrent Memory with Optimal Polynomial Projections
2020/08/17 by Albert Gu, Tri Dao, Gu, Albert +7 · 77 citations
Computer Science · #Time Series Analysis and Forecasting #Anomaly Detection Techniques and Applications #Data Stream Mining Techniques
- Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers
2021/10/26 by Gu, Albert, Johnson, Isys, Goel, Karan +4 · 79 citations
#Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG)
- Simple linear attention language models balance the recall-throughput tradeoff
2024/02/28 by Simran Arora, Arora, Simran, Sabri Eyuboglu +15 · 35 citations
Computer Science · #Topic Modeling #Domain Adaptation and Few-Shot Learning #Advanced Graph Neural Networks
- How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections
2022/06/24 by Albert Gu, Isys Johnson, Gu, Albert +7 · 22 citations
Computer Science · #Blind Source Separation Techniques #Neural Networks and Applications #Target Tracking and Data Fusion in Sensor Networks
- Zoology: Measuring and Improving Recall in Efficient Language Models
2023/12/08 by Simran Arora, Sabri Eyuboglu, Arora, Simran +13 · 26 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Multimodal Machine Learning Applications #Topic Modeling
- FAQ: Questions Asked Frequently
2015/04/15 by Mahmoud Abo Khamis, Hung Q. Ngo, Khamis, Mahmoud Abo +3 · 10 citations
Computer Science · #Bayesian Modeling and Causal Inference #Constraint Satisfaction and Optimization #Data Management and Algorithms #Data Structures and Algorithms (cs.DS) #Databases (cs.DB) #FOS: Computer and information sciences #Logic in Computer Science (cs.LO)
- Skew Strikes Back: New Developments in the Theory of Join Algorithms
2013/10/12 by Hung Q. Ngo, Christopher Ré, Ngo, Hung Q. +3 · 10 citations
Computer Science · #Advanced Database Systems and Queries #Data Management and Algorithms #Data Structures and Algorithms (cs.DS) #Databases (cs.DB) #FOS: Computer and information sciences #Optimization and Search Problems
- Explicit Codes Achieving List Decoding Capacity: Error-correction with Optimal Redundancy
2005/11/18 by Guruswami, Venkatesan, Rudra, Atri · 7 citations
#FOS: Computer and information sciences #H.1.1 #Information Theory (cs.IT)
- Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations
2019/03/14 by Dao, Tri, Gu, Albert, Eichhorn, Matthew +2 · 9 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)
- Scatterbrain: Unifying Sparse and Low-rank Attention Approximation
2021/10/28 by Chen, Beidi, Dao, Tri, Winsor, Eric +3 · 10 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG)
- Monarch: Expressive Structured Matrices for Efficient and Accurate Training
2022/04/01 by Tri Dao, Dao, Tri, Beidi Chen +17 · 9 citations
Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Parallel Computing and Optimization Techniques
- Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models
2021/11/30 by Tri Dao, Dao, Tri, Beidi Chen +11 · 7 citations
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications
- Just read twice: closing the recall gap for recurrent language models
2024/07/07 by Arora, Simran, Timalsina, Aman, Singhal, Aaryan +6 · 11 citations
#Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG)
- Cartridges: Lightweight and general-purpose long context representations via self-study
2025/06/06 by Sabri Eyuboglu, Ryan Ehrlich, Eyuboglu, Sabri +18 · 20 citations
Computer Science · Social Sciences · #Topic Modeling #Natural Language Processing Techniques #Computational and Text Analysis Methods
- Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
2023/10/18 by Daniel Y. Fu, Simran Arora, Fu, Daniel Y. +17 · 7 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Neural Networks and Applications #Topic Modeling
- Simple Hardware-Efficient Long Convolutions for Sequence Modeling
2023/02/13 by Fu, Daniel Y., Epstein, Elliot L., Nguyen, Eric +5 · 5 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG)
- Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions
2023/10/28 by Stefano Massaroli, Michael Poli, Massaroli, Stefano +23 · 5 citations
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Signal Processing (eess.SP) #Topic Modeling #electronic engineering #information engineering
- Average-radius list-recovery of random linear codes: it really ties the\n room together
2017/04/07 by Atri Rudra, Mary Wootters, Rudra, Atri +1 · 2 citations
Computer Science · #Coding theory and cryptography #Cooperative Communication and Network Coding #Cryptography and Data Security #Error Correcting Code Techniques #FOS: Computer and information sciences #Information Theory (cs.IT)
- Learning Compressed Transforms with Low Displacement Rank
2018/10/04 by Thomas, Anna T., Gu, Albert, Dao, Tri +2 · 2 citations
#FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)
- When LP is the Cure for Your Matching Woes: Improved Bounds for Stochastic Matchings
2010/08/31 by Nikhil Bansal, Anupam Gupta, Bansal, Nikhil +9 · 1 citation
Computer Science · Decision Sciences · #Auction Theory and Applications #Complexity and Algorithms in Graphs #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #Optimization and Search Problems
- Every list-decodable code for high noise has abundant near-optimal rate\n puncturings
2013/10/07 by Atri Rudra, Rudra, Atri, Mary Wootters +1 · 1 citation
Computer Science · Engineering · #Coding theory and cryptography #Cooperative Communication and Network Coding #FOS: Computer and information sciences #Information Theory (cs.IT) #graph theory and CDMA systems
- Joins via Geometric Resolutions: Worst-case and Beyond
2014/04/02 by Khamis, Mahmoud Abo, Ngo, Hung Q., Ré, Christopher +1 · 1 citation
#Data Structures and Algorithms (cs.DS) #Databases (cs.DB) #FOS: Computer and information sciences
- It'll probably work out: improved list-decoding through random\n operations
2014/08/10 by Atri Rudra, Mary Wootters, Rudra, Atri +1 · 1 citation
Biochemistry, Genetics and Molecular Biology · Computer Science · Engineering · #Coding theory and cryptography #DNA and Biological Computing #Error Correcting Code Techniques #FOS: Computer and information sciences #Information Theory (cs.IT) #graph theory and CDMA systems
- Join Processing for Graph Patterns: An Old Dog with New Tricks
2015/03/13 by Nguyen, Dung, Aref, Molham, Bravenboer, Martin +4 · 1 citation
#Data Structures and Algorithms (cs.DS) #Databases (cs.DB) #E.1 #FOS: Computer and information sciences #H.2 #H.3.4
- The Range of Topological Effects on Communication
2015/04/24 by Arkadev Chattopadhyay, Atri Rudra, Chattopadhyay, Arkadev +1 · 1 citation
Computer Science · #Advanced Graph Theory Research #Complexity and Algorithms in Graphs #Computational Complexity (cs.CC) #FOS: Computer and information sciences #Interconnection Networks and Systems
- Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps
2020/12/29 by Tri Dao, Nimit S. Sohoni, Dao, Tri +13 · 1 citation
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Speech Recognition and Synthesis #Stochastic Gradient Optimization Techniques