Yuanzhi Li
- Sparks of Artificial General Intelligence: Early experiments with GPT-4
2023/03/22 by Sébastien Bubeck, Bubeck, Sébastien, Varun Chandrasekaran +26 · 21 voices · 276 citations
Medicine · Computer Science · #Artificial Intelligence in Healthcare and Education #Topic Modeling #Machine Learning in Healthcare
- TinyStories: How Small Can Language Models Be and Still Speak Coherent English?
2023/05/12 by Ronen Eldan, Eldan, Ronen, Yuanzhi Li +1 · 19 voices · 77 citations
Computer Science · #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling #cs.AI #cs.CL #cs.LG
- LoRA: Low-Rank Adaptation of Large Language Models
2021/06/17 by J. Edward Hu, Edward J. Hu, Yelong Shen +14 · 10 voices · 2069 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Speech Recognition and Synthesis
- Textbooks Are All You Need
2023/06/20 by Suriya Gunasekar, Gunasekar, Suriya, Yi Zhang +35 · 11 voices · 55 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Text Readability and Simplification
- Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone
2024/04/22 by Marah Abdin, Abdin, Marah, Jyoti Aneja +262 · 8 voices · 308 citations
Computer Science · #Computational Physics and Python Applications
- Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine
2023/11/28 by Harsha Nori, Nori, Harsha, Yin Tat Lee +36 · 5 voices · 47 citations
Computer Science · Medicine · #Artificial Intelligence in Healthcare and Education #Machine Learning in Healthcare #Topic Modeling #cs.CL
- Phi-4 Technical Report
2024/12/12 by Marah Abdin, Jyoti Aneja, Abdin, Marah +51 · 4 voices · 175 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.AI #cs.CL
- Textbooks Are All You Need II: phi-1.5 technical report
2023/09/11 by Yuanzhi Li, Sébastien Bubeck, Li, Yuanzhi +9 · 3 voices · 48 citations
Computer Science · #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling #cs.AI #cs.CL
- Positional Description Matters for Transformers Arithmetic
2023/11/22 by Ruoqi Shen, Sébastien Bubeck, Shen, Ruoqi +9 · 1 voice · 10 citations
Computer Science · #Numerical Methods and Algorithms #Topic Modeling #Mathematics, Computing, and Information Processing
- A Convergence Theory for Deep Learning via Over-Parameterization
2018/11/09 by Zeyuan Allen-Zhu, Allen-Zhu, Zeyuan, Yuanzhi Li +3 · 118 citations
Computer Science · Physics and Astronomy · #Reinforcement Learning in Robotics #Model Reduction and Neural Networks #Evolutionary Algorithms and Applications
- Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
2023/09/25 by Zeyuan Allen Zhu, Zeyuan Allen-Zhu, Allen-Zhu, Zeyuan +2 · 4 voices · 48 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.LG
- Gradient Descent on Neural Networks Typically Occurs at the Edge of\n Stability
2021/02/26 by Jeremy M. Cohen, Cohen, Jeremy M., Simran Kaur +7 · 42 citations
Computer Science · Physics and Astronomy · #Stochastic Gradient Optimization Techniques #Neural Networks and Applications #Model Reduction and Neural Networks
- Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws
2024/04/08 by Zeyuan Allen-Zhu, Allen-Zhu, Zeyuan, Yuanzhi Li +1 · 2 voices · 16 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.LG
- Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning
2020/12/17 by Zeyuan Allen-Zhu, Allen-Zhu, Zeyuan, Yuanzhi Li +1 · 25 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE) #Optimization and Control (math.OC)
- Learning Overparameterized Neural Networks via Stochastic Gradient\n Descent on Structured Data
2018/08/03 by Yuanzhi Li, Yingyu Liang, Li, Yuanzhi +1 · 31 citations
Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and ELM #Neural Networks and Applications
- The probability flow ODE is provably fast
2023/05/19 by Sitan Chen, Sinho Chewi, Chen, Sitan +9 · 23 citations
Computer Science · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Statistics Theory (math.ST)
- SmartPlay: A Benchmark for LLMs as Intelligent Agents
2023/10/02 by Yue Wu, Wu, Yue, Xuan Tang +5 · 22 citations
Computer Science · #Artificial Intelligence (cs.AI) #Digital Rights Management and Security #FOS: Computer and information sciences #Machine Learning (cs.LG) #Mobile Agent-Based Network Management #Peer-to-Peer Network Technologies
- Convergence Analysis of Two-layer Neural Networks with ReLU Activation
2017/05/28 by Yuanzhi Li, Yuan Yang, Li, Yuanzhi +1 · 15 citations
Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and ELM #Stochastic Gradient Optimization Techniques
- Mixture of Parrots: Experts improve memorization more than reasoning
2024/10/24 by Samy Jelassi, Clara Mohri, Jelassi, Samy +17 · 2 voices · 7 citations
Agricultural and Biological Sciences · Psychology · #Animal Behavior and Reproduction #Primate Behavior and Ecology
- Physics of Language Models: Part 1, Learning Hierarchical Language Structures
2023/05/23 by Zeyuan Allen-Zhu, Yuanzhi Li, Allen-Zhu, Zeyuan +1 · 17 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Natural Language Processing Techniques #Topic Modeling
- OpenAI GPT-5 System Card
2025/12/19 by Aaditya Singh, Adam Fry, Adam Perelman +483 · 2 voices · 36 citations
#cs.CL #cs.AI
- Physics of Language Models: Part 3.2, Knowledge Manipulation
2023/09/25 by Zeyuan Allen-Zhu, Yuanzhi Li, Allen-Zhu, Zeyuan +1 · 15 citations
Computer Science · #Topic Modeling #Machine Learning and Algorithms #Explainable Artificial Intelligence (XAI)
- Toward Understanding the Feature Learning Process of Self-supervised\n Contrastive Learning
2021/05/31 by Zixin Wen, Yuanzhi Li, Wen, Zixin +1 · 9 citations
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification
- Neon2: Finding Local Minima via First-Order Oracles
2017/11/17 by Zeyuan Allen-Zhu, Yuanzhi Li, Allen-Zhu, Zeyuan +1 · 10 citations
Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Complexity and Algorithms in Graphs
- O1 Replication Journey: A Strategic Progress Report -- Part 1
2024/10/08 by Yiwei Qin, Qin, Yiwei, Xuefeng Li +18 · 17 citations
Engineering · Computer Science · #Spacecraft and Cryogenic Technologies #Distributed and Parallel Computing Systems #Service-Oriented Architecture and Web Services
- How Do Transformers Learn Topic Structure: Towards a Mechanistic Understanding
2023/03/07 by Yuchen Li, Yuanzhi Li, Li, Yuchen +3 · 9 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Text and Document Classification Technologies
- An Alternative View: When Does SGD Escape Local Minima?
2018/02/17 by Robert Kleinberg, Yuanzhi Li, Kleinberg, Robert +3 · 5 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine Learning and ELM #Stochastic Gradient Optimization Techniques
- What Can ResNet Learn Efficiently, Going Beyond Kernels?
2019/05/24 by Zeyuan Allen-Zhu, Yuanzhi Li, Allen-Zhu, Zeyuan +1 · 8 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Data Structures and Algorithms (cs.DS) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE) #Optimization and Control (math.OC)
- LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
2024/10/16 by Akshara Prabhakar, Yuanzhi Li, Prabhakar, Akshara +9 · 13 citations
Computer Science · Engineering · #Computation and Language (cs.CL) #Context-Aware Activity Recognition Systems #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning #Machine Learning (cs.LG) #Robotics and Automated Systems
- Complexity of Highly Parallel Non-Smooth Convex Optimization
2019/06/25 by Sébastien Bubeck, Bubeck, Sébastien, Qijia Jiang +7 · 4 citations
Computer Science · Engineering · #Complexity and Algorithms in Graphs #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Near-Optimal Discrete Optimization for Experimental Design: A Regret Minimization Approach
2017/11/14 by Zeyuan Allen-Zhu, Yuanzhi Li, Allen-Zhu, Zeyuan +5 · 3 citations
Computer Science · Engineering · #Computation (stat.CO) #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Sparse and Compressive Sensing Techniques
- Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems
2024/08/29 by Ye Tian, Ye, Tian, Zicheng Xu +5 · 9 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques
- Near-optimal discrete optimization for experimental design: a regret minimization approach
2020/01/10 by Zeyuan Allen-Zhu, Yuanzhi Li, Aarti Singh +1 · 4 citations
Engineering · Computer Science · #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #Machine Learning and Algorithms
- TinyGSM: achieving >80% on GSM8k with small language models
2023/12/14 by Bingbin Liu, Sébastien Bubeck, Liu, Bingbin +13 · 5 citations
Computer Science · Social Sciences · #Computation and Language (cs.CL) #Educational Assessment and Pedagogy #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Text Readability and Simplification
- Length Generalization in Arithmetic Transformers
2023/06/27 by Samy Jelassi, Stéphane d’Ascoli, Jelassi, Samy +9 · 4 citations
Computer Science · Physics and Astronomy · #Computational Physics and Python Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural Networks and Applications
- First Efficient Convergence for Streaming k-PCA: a Global, Gap-Free, and Near-Optimal Rate
2016/07/26 by Zeyuan Allen-Zhu, Yuanzhi Li, Allen-Zhu, Zeyuan +1 · 3 citations
Computer Science · Engineering · Mathematics · #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Random Matrices and Applications #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- How Does Adaptive Optimization Impact Local Neural Network Geometry?
2022/11/04 by Kaiqi Jiang, Jiang, Kaiqi, Dhruv Malik +3 · 3 citations
Computer Science · Engineering · #Advanced Image Processing Techniques #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- LLM360 K2: Building a 65B 360-Open-Source Large Language Model from Scratch
2025/01/13 by Zhengzhong Liu, Bowen Tan, Liu, Zhengzhong +47 · 7 citations
Computer Science · #Natural Language Processing Techniques
- Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization
2021/08/25 by Difan Zou, Zou, Difan, Yuan Cao +5 · 2 citations
Computer Science · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Neural Networks and Applications #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
- Backward Feature Correction: How Deep Learning Performs Deep (Hierarchical) Learning
2020/01/13 by Zeyuan Allen-Zhu, Allen-Zhu, Zeyuan, Yuanzhi Li +1 · 2 citations
Computer Science · #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE) #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
- Improved Path-length Regret Bounds for Bandits
2019/01/29 by Sébastien Bubeck, Yuanzhi Li, Bubeck, Sébastien +5 · 2 citations
Computer Science · Decision Sciences · Engineering · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Smart Grid Energy Management
- LazySVD: Even Faster SVD Decomposition Yet Without Agonizing Pain
2016/07/12 by Zeyuan Allen-Zhu, Allen-Zhu, Zeyuan, Yuanzhi Li +1 · 1 citation
Computer Science · Engineering · #Blind Source Separation Techniques #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Follow the Compressed Leader: Faster Online Learning of Eigenvectors and Faster MMWU
2017/01/06 by Zeyuan Allen-Zhu, Yuanzhi Li, Allen-Zhu, Zeyuan +1 · 1 citation
Computer Science · Decision Sciences · #Stochastic Gradient Optimization Techniques #Advanced Bandit Algorithms Research #Optimization and Search Problems
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data
2024/10/11 by Binghui Li, Li, Binghui, Yuanzhi Li +1 · 3 citations
Computer Science · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG)
- When Is Generalizable Reinforcement Learning Tractable?
2021/01/01 by Dhruv Malik, Malik, Dhruv, Yuanzhi Li +3 · 1 citation
Computer Science · #Reinforcement Learning in Robotics #Machine Learning and Algorithms #Computability, Logic, AI Algorithms
- Dissecting adaptive methods in GANs
2022/10/09 by Samy Jelassi, Jelassi, Samy, David Dobre +7 · 1 citation
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Music and Audio Processing
- Doubly Accelerated Methods for Faster CCA and Generalized Eigendecomposition
2016/07/20 by Zeyuan Allen-Zhu, Allen-Zhu, Zeyuan, Yuanzhi Li +1 · 1 citation
Computer Science · Engineering · Neuroscience · #Blind Source Separation Techniques #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Face and Expression Recognition #Functional Brain Connectivity Studies #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques
- SPRING: Studying the Paper and Reasoning to Play Games
2023/05/24 by Yue Wu, Shrimai Prabhumoye, Wu, Yue +13 · 1 citation
Computer Science · Psychology · #Artificial Intelligence (cs.AI) #Artificial Intelligence in Games #Educational Games and Gamification #FOS: Computer and information sciences #Machine Learning (cs.LG) #Topic Modeling
- Provably learning a multi-head attention layer
2024/02/06 by Sitan Chen, Chen, Sitan, Yuanzhi Li +1 · 1 citation
Computer Science · Neuroscience · #Advanced Neural Network Applications #Brain Tumor Detection and Classification #Data Structures and Algorithms (cs.DS) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)
- Role of Locality and Weight Sharing in Image-Based Tasks: A Sample Complexity Separation between CNNs, LCNs, and FCNs
2024/03/23 by Aakash Lahoti, Stefani Karp, Lahoti, Aakash +7 · 1 citation
Neuroscience · Computer Science · #Brain Tumor Detection and Classification #Advanced Neural Network Applications
- AgentKit: Structured LLM Reasoning with Dynamic Graphs
2024/04/17 by Yue Wu, Wu, Yue, Yewen Fan +15 · 1 citation
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #Auction Theory and Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multi-Agent Systems and Negotiation #Peer-to-Peer Network Technologies