Sham M. Kakade
- Matryoshka Representation Learning
2022/05/26 by Aditya Kusupati, Gantavya Bhatt, Kusupati, Aditya +20 · 4 voices · 55 citations
Computer Science · Medicine · #cs.LG #cs.CV
- Transcendence: Generative Models Can Outperform The Experts That Train Them
2024/06/17 by Edwin Zhang, Vincent Zhu, Zhang, Edwin +13 · 8 voices · 9 citations
#cs.LG #cs.AI
- DataComp-LM: In search of the next generation of training sets for language models
2024/06/17 by Jeffrey Li, Alex Chengyu Fang, Li, Jeffrey +121 · 2 voices · 78 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.LG
- Provably Efficient Maximum Entropy Exploration
2018/12/06 by Elad Hazan, Sham M. Kakade, Hazan, Elad +5 · 1 voice · 29 citations
Computer Science · Engineering · Mathematics · #CCD and CMOS Imaging Sensors #Image and Signal Denoising Methods #Sparse and Compressive Sensing Techniques #cs.AI #cs.LG #stat.ML
- MatFormer: Nested Transformer for Elastic Inference
2023/10/11 by Devvrit, Sneha Kudugunta, Aditya Kusupati +20 · 5 voices · 19 citations
Computer Science · Social Sciences · #Caching and Content Delivery #Human Mobility and Location-Based Analysis #Opportunistic and Delay-Tolerant Networks #cs.CL #cs.CV #cs.LG
- Global Convergence of Policy Gradient Methods for the Linear Quadratic Regulator
2018/01/15 by Maryam Fazel, Fazel, Maryam, Rong Ge +5 · 41 citations
Computer Science · Engineering · Physics and Astronomy · #Adaptive Dynamic Programming Control #Advanced Control Systems Optimization #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks
- Meta-Learning with Implicit Gradients
2019/09/10 by Aravind Rajeswaran, Rajeswaran, Aravind, Chelsea Finn +5 · 58 citations
Computer Science · Physics and Astronomy · #Domain Adaptation and Few-Shot Learning #Advanced Neural Network Applications #Model Reduction and Neural Networks
- SOAP: Improving and Stabilizing Shampoo using Adam
2024/09/17 by Nikhil Vyas, Depen Morwani, Vyas, Nikhil +14 · 2 voices · 42 citations
Engineering · Computer Science · Biochemistry, Genetics and Molecular Biology · #Tree Root and Stability Studies #Music Technology and Sound Studies #Animal Vocal Communication and Behavior
- Efficient Learning of Generalized Linear and Single Index Models with\n Isotonic Regression
2011/04/11 by Sham M. Kakade, Kakade, Sham, Adam Tauman Kalai +5 · 15 citations
Mathematics · Computer Science · #Statistical Methods and Inference #Bayesian Modeling and Causal Inference #Bayesian Methods and Mixture Models
- Repeat After Me: Transformers are Better than State Space Models at Copying
2024/02/01 by Samy Jelassi, David Brandfonbrener, Jelassi, Samy +5 · 35 citations
Decision Sciences · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Scientific Computing and Data Management #Simulation Techniques and Applications
- Robust Aggregation for Federated Learning
2022/01/01 by Krishna Pillutla, Sham M. Kakade, Zaïd Harchaoui +1 · 22 citations
Computer Science · #Cryptography and Data Security #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques
- Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Control
2018/11/05 by Kendall Lowrey, Lowrey, Kendall, Aravind Rajeswaran +7 · 22 citations
Computer Science · Physics and Astronomy · Decision Sciences · #Reinforcement Learning in Robotics #Model Reduction and Neural Networks #Simulation Techniques and Applications
- Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions
2025/02/10 by Jaeyeon Kim, Kim, Jaeyeon, Kulin Shah +7 · 63 citations
Engineering · #Architecture and Computational Design
- The Statistical Complexity of Interactive Decision Making
2021/12/27 by Dylan J. Foster, Sham M. Kakade, Foster, Dylan J. +5 · 1 voice · 13 citations
Computer Science · Decision Sciences · Mathematics · #Advanced Bandit Algorithms Research #Data Stream Mining Techniques #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Optimization and Control (math.OC) #Statistics Theory (math.ST) #cs.LG #math.OC #math.ST #stat.ML
- Revisiting the Polyak step size
2019/05/01 by Elad Hazan, Sham M. Kakade, Hazan, Elad +1 · 13 citations
Computer Science · Decision Sciences · Engineering · #Advanced Bandit Algorithms Research #FOS: Mathematics #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Mixture of Parrots: Experts improve memorization more than reasoning
2024/10/24 by Samy Jelassi, Clara Mohri, Jelassi, Samy +17 · 2 voices · 7 citations
Agricultural and Biological Sciences · Psychology · #Animal Behavior and Reproduction #Primate Behavior and Ecology
- Opponent interactions between serotonin and dopamine
2002/06/01 by Nathaniel D. Daw, Nathaniel D Daw, Sham Kakade +2 · 8 citations
Neuroscience · Biochemistry, Genetics and Molecular Biology · #Neurotransmitter Receptor Influence on Behavior #Receptor Mechanisms and Signaling #Neural and Behavioral Psychology Studies
- A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Norm
2019/02/11 by Chi Jin, Praneeth Netrapalli, Jin, Chi +7 · 10 citations
Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Mathematical Approximation and Integration #Point processes and geometric inequalities #Probability (math.PR) #Random Matrices and Applications
- Few-Shot Learning via Learning the Representation, Provably
2020/02/21 by Simon S. Du, Wei Hu, Du, Simon S. +7 · 20 citations
Computer Science · Engineering · #Domain Adaptation and Few-Shot Learning #Machine Learning and ELM #Sparse and Compressive Sensing Techniques
- Multi-Label Prediction via Compressed Sensing
2009/02/08 by Daniel Hsu, Hsu, Daniel, Sham M. Kakade +5 · 8 citations
Computer Science · Decision Sciences · Engineering · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Sparse and Compressive Sensing Techniques
- Hidden Progress in Deep Learning: SGD Learns Parities Near the Computational Limit
2022/07/18 by Boaz Barak, Barak, Boaz, Benjamin Edelman +9 · 13 citations
Computer Science · Materials Science · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning in Materials Science #Neural and Evolutionary Computing (cs.NE) #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
- Online Meta-Learning
2019/02/22 by Chelsea Finn, Finn, Chelsea, Aravind Rajeswaran +5 · 14 citations
Computer Science · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Healthcare #Multimodal Machine Learning Applications
- A tail inequality for quadratic forms of subgaussian random vectors
2011/10/13 by Daniel Hsu, Sham M. Kakade, Hsu, Daniel +3 · 6 citations
Mathematics · #Approximation Theory and Sequence Spaces #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Mathematical Inequalities and Applications #Point processes and geometric inequalities #Probability (math.PR)
- Inductive Biases and Variable Creation in Self-Attention Mechanisms
2021/10/19 by Benjamin Edelman, Surbhi Goel, Edelman, Benjamin L. +5 · 10 citations
Computer Science · #Data Stream Mining Techniques #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Topic Modeling
- Learning mixtures of spherical Gaussians: moment methods and spectral decompositions
2012/06/25 by Daniel Hsu, Hsu, Daniel, Sham M. Kakade +1 · 6 citations
Computer Science · Chemistry · #Blind Source Separation Techniques #Bayesian Methods and Mixture Models #Spectroscopy and Chemometric Analyses
- Unpacking Reward Shaping: Understanding the Benefits of Reward Engineering on Sample Complexity
2022/10/18 by Abhishek Gupta, Gupta, Abhishek, Aldo Pacchiano +7 · 10 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Software Engineering Research
- Scaling Laws in Linear Regression: Compute, Parameters, and Data
2024/06/12 by Licong Lin, Jingfeng Wu, Lin, Licong +7 · 15 citations
Mathematics · #Advanced Statistical Methods and Models #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistics Theory (math.ST)
- Optimal Regularization Can Mitigate Double Descent
2020/03/04 by Preetum Nakkiran, Nakkiran, Preetum, Prayaag Venkat +5 · 7 citations
Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE) #Sparse and Compressive Sensing Techniques #Statistics Theory (math.ST) #Stochastic Gradient Optimization Techniques
- Online Control with Adversarial Disturbances
2019/02/23 by Naman Agarwal, Brian Bullins, Agarwal, Naman +7 · 10 citations
Decision Sciences · Engineering · Physics and Astronomy · #Advanced Bandit Algorithms Research #Advanced Control Systems Optimization #FOS: Computer and information sciences #FOS: Electrical engineering #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Model Reduction and Neural Networks #Optimization and Control (math.OC) #Systems and Control (eess.SY) #electronic engineering #information engineering
- Learning Features of Music from Scratch
2016/11/29 by John Thickstun, Zaïd Harchaoui, Thickstun, John +3 · 8 citations
Computer Science · #Music and Audio Processing #Music Technology and Sound Studies #Speech and Audio Processing
- PC-PG: Policy Cover Directed Exploration for Provable Policy Gradient Learning
2020/07/16 by Alekh Agarwal, Agarwal, Alekh, Mikael Henaff +5 · 8 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics
- What are the Statistical Limits of Offline RL with Linear Function Approximation?
2020/10/22 by Ruosong Wang, Wang, Ruosong, Dean P. Foster +3 · 12 citations
Computer Science · Decision Sciences · #Reinforcement Learning in Robotics #Machine Learning and Algorithms #Advanced Bandit Algorithms Research
- Is a Good Representation Sufficient for Sample Efficient Reinforcement\n Learning?
2019/10/07 by Simon S. Du, Sham M. Kakade, Du, Simon S. +5 · 8 citations
Computer Science · Decision Sciences · #Adaptive Dynamic Programming Control #Advanced Bandit Algorithms Research #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Optimization and Control (math.OC) #Reinforcement Learning in Robotics
- Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems
2024/02/27 by Zhenting Qi, Hanlin Zhang, Qi, Zhenting +7 · 12 citations
Computer Science · #Algorithms and Data Compression #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Cryptography and Security (cs.CR) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Semantic Web and Ontologies #Web Data Mining and Analysis
- A Spectral Algorithm for Latent Dirichlet Allocation
2012/04/30 by Animashree Anandkumar, Dean P. Foster, Anandkumar, Animashree +7 · 7 citations
Computer Science · #Bayesian Methods and Mixture Models #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Text and Document Classification Technologies #Topic Modeling
- Soft Threshold Weight Reparameterization for Learnable Sparsity
2020/02/08 by Aditya Kusupati, Kusupati, Aditya, Vivek Ramanujan +11 · 7 citations
Computer Science · Engineering · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Infrastructure Maintenance and Monitoring #Machine Learning (cs.LG) #Machine Learning (stat.ML)
- Model-Based Reinforcement Learning with a Generative Model is Minimax\n Optimal
2019/06/10 by Alekh Agarwal, Agarwal, Alekh, Sham M. Kakade +3 · 9 citations
Computer Science · #Reinforcement Learning in Robotics #Explainable Artificial Intelligence (XAI)
- The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning\n Rate Procedure For Least Squares
2019/04/29 by Rong Ge, Ge, Rong, Sham M. Kakade +5 · 5 citations
Computer Science · Engineering · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Markov Chains and Monte Carlo Methods #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks
2024/10/16 by Akshara Prabhakar, Yuanzhi Li, Prabhakar, Akshara +9 · 13 citations
Computer Science · Engineering · #Computation and Language (cs.CL) #Context-Aware Activity Recognition Systems #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning #Machine Learning (cs.LG) #Robotics and Automated Systems
- Understanding Contrastive Learning Requires Incorporating Inductive Biases
2022/02/28 by Nikunj Saunshi, Jordan T. Ash, Saunshi, Nikunj +13 · 6 citations
Computer Science · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine Learning and Data Classification
- Sample-Efficient Reinforcement Learning of Undercomplete POMDPs
2020/06/22 by Chi Jin, Sham M. Kakade, Jin, Chi +5 · 5 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #Blind Source Separation Techniques #Elevator Systems and Control #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Optimization and Control (math.OC)
- How Does Critical Batch Size Scale in Pre-training?
2024/10/29 by Hanlin Zhang, Zhang, Hanlin, Depen Morwani +13 · 14 citations
Psychology · #Human Resource Development and Performance Evaluation
- Un-regularizing: approximate proximal point and faster stochastic\n algorithms for empirical risk minimization
2015/06/24 by Roy Frostig, Rong Ge, Frostig, Roy +5 · 5 citations
Computer Science · Decision Sciences · Engineering · #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Risk and Portfolio Optimization #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Interpreting the linear structure of vision-language model embedding spaces
2025/04/16 by Isabel Papadimitriou, Huangyuan Su, Papadimitriou, Isabel +8 · 2 voices · 11 citations
Computer Science · Social Sciences · Psychology · #Multimodal Machine Learning Applications #Language and cultural evolution #Language, Metaphor, and Cognition
- A New Perspective on Shampoo's Preconditioner
2024/06/25 by Depen Morwani, Morwani, Depen, Itai Shapira +9 · 10 citations
Engineering · #Acoustic Wave Phenomena Research #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Vibration Control and Rheological Fluids
- Playing Games with Approximation Algorithms
2009/01/01 by Sham M. Kakade, Adam Tauman Kalai, Katrina Ligett · 3 citations
Decision Sciences · Computer Science · #Advanced Bandit Algorithms Research #Optimization and Search Problems #Machine Learning and Algorithms
- Robust and Differentially Private Mean Estimation
2021/02/18 by Xiyang Liu, Weihao Kong, Liu, Xiyang +5 · 4 citations
Computer Science · #Adversarial Robustness in Machine Learning #Cryptography and Security (cs.CR) #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Privacy-Preserving Technologies in Data
- Calibration, Entropy Rates, and Memory in Language Models
2019/06/11 by Mark Braverman, Braverman, Mark, Xinyi Chen +9 · 4 citations
Computer Science · Social Sciences · #Topic Modeling #Natural Language Processing Techniques #Language and cultural evolution
- Gone Fishing: Neural Active Learning with Fisher Embeddings
2021/06/17 by Jordan T. Ash, Surbhi Goel, Ash, Jordan T. +5 · 4 citations
Computer Science · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Algorithms #Machine Learning and Data Classification
- Benign Overfitting of Constant-Stepsize SGD for Linear Regression
2021/03/23 by Difan Zou, Jingfeng Wu, Zou, Difan +7 · 4 citations
Computer Science · Engineering · #3D Shape Modeling and Analysis #FOS: Computer and information sciences #FOS: Mathematics #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Stochastic Gradient Optimization Techniques
- Random Scaling of Emergent Capabilities
2025/02/24 by Rosie Zhao, Zhao, Rosie, Tian Qin +8 · 2 voices · 5 citations
Decision Sciences · #Complex Systems and Decision Making #cs.LG
- AdANNS: A Framework for Adaptive Semantic Search
2023/05/30 by Aniket Rege, Rege, Aniket, Aditya Kusupati +14 · 1 voice · 2 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Image Retrieval and Classification Techniques #Information Retrieval (cs.IR) #Machine Learning (cs.LG) #cs.IR #cs.LG
- Streaming PCA: Matching Matrix Bernstein and Near-Optimal Finite Sample Guarantees for Oja's Algorithm
2016/02/22 by Prateek Jain, Chi Jin, Jain, Prateek +7 · 2 citations
Computer Science · Engineering · Mathematics · #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE) #Random Matrices and Applications #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- From an Image to a Scene: Learning to Imagine the World from a Million 360 Videos
2024/12/10 by Matthew Wallingford, Wallingford, Matthew, Anand Bhattad +17 · 7 citations
Health Professions · Social Sciences · #Digital Storytelling and Education #Educator Training and Historical Pedagogy
- Prediction with a Short Memory
2016/12/08 by Vatsal Sharan, Sharan, Vatsal, Sham M. Kakade +5 · 2 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Algorithms and Data Compression #Artificial Intelligence (cs.AI) #Computational Complexity (cs.CC) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms
- Universal Length Generalization with Turing Programs
2024/07/03 by Kaiying Hou, Hou, Kaiying, David Brandfonbrener +7 · 5 citations
Computer Science · #Computability, Logic, AI Algorithms #FOS: Computer and information sciences #Machine Learning (cs.LG)
- The Power and Limitation of Pretraining-Finetuning for Linear Regression under Covariate Shift
2022/08/03 by Jingfeng Wu, Difan Zou, Wu, Jingfeng +7 · 3 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Optimization and Control (math.OC)
- The Nonstochastic Control Problem
2019/11/27 by Elad Hazan, Hazan, Elad, Sham M. Kakade +3 · 2 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Search Problems #Reinforcement Learning in Robotics
- An Exponential Lower Bound for Linearly-Realizable MDPs with Constant Suboptimality Gap
2021/03/23 by Yuanhao Wang, Ruosong Wang, Wang, Yuanhao +3 · 3 citations
Decision Sciences · Computer Science · #Advanced Bandit Algorithms Research #Reinforcement Learning in Robotics #Machine Learning and Algorithms
- Deconstructing What Makes a Good Optimizer for Language Models
2024/07/10 by Rosie Zhao, Depen Morwani, Zhao, Rosie +7 · 6 citations
Computer Science · #Natural Language Processing Techniques
- Feature emergence via margin maximization: case studies in algebraic tasks
2023/11/13 by Depen Morwani, Benjamin Edelman, Morwani, Depen +7 · 3 citations
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #I.2.6 #I.5.1 #Machine Learning (cs.LG) #Model Reduction and Neural Networks #Neural Networks and Applications #Topological and Geometric Data Analysis
- Multi-Stage Episodic Control for Strategic Exploration in Text Games
2022/01/04 by Jens Tuyls, Tuyls, Jens, Shunyu Yao +5 · 2 citations
Computer Science · Psychology · #Artificial Intelligence in Games #Computation and Language (cs.CL) #Educational Games and Gamification #FOS: Computer and information sciences #Reinforcement Learning in Robotics
- A Study on the Calibration of In-context Learning
2023/12/07 by Hanlin Zhang, Zhang, Hanlin, Yifan Zhang +13 · 3 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Education and Learning Interventions #FOS: Computer and information sciences #Machine Learning (cs.LG)
- Faster Eigenvector Computation via Shift-and-Invert Preconditioning
2016/05/26 by Dan Garber, Elad Hazan, Garber, Dan +11 · 2 citations
Computer Science · Engineering · #Data Structures and Algorithms (cs.DS) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Matrix Theory and Algorithms #Numerical Analysis (math.NA) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- CoLoR-Filter: Conditional Loss Reduction Filtering for Targeted Language Model Pre-training
2024/06/15 by David Brandfonbrener, Hanlin Zhang, Brandfonbrener, David +7 · 4 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Speech Recognition and Synthesis
- Regularization Techniques for Learning with Matrices
2009/10/04 by Sham M. Kakade, Shai Shalev‐Shwartz, Kakade, Sham M. +3 · 1 citation
Computer Science · Engineering · #FOS: Computer and information sciences #Face and Expression Recognition #I.2.6 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Sparse and Compressive Sensing Techniques
- Any-Order Flexible Length Masked Diffusion
2025/08/31 by J.M. Kim, Lee Cheuk-Kit, Kim, Jaeyeon +13 · 12 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Machine Learning in Healthcare
- Cognitive models can reveal interpretable value trade-offs in language models
2025/06/25 by Sonia K. Murthy, Rosie Zhao, Murthy, Sonia K. +13 · 3 voices
Computer Science · #Natural Language Processing Techniques #Semantic Web and Ontologies
- Provably Correct Automatic Subdifferentiation for Qualified Programs
2018/09/23 by Sham M. Kakade, Jason D. Lee, Kakade, Sham +1 · 1 citation
Computer Science · #Advanced Multi-Objective Optimization Algorithms #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Numerical Methods and Algorithms #Optimization and Control (math.OC)
- Maximum Likelihood Estimation for Learning Populations of Parameters
2019/02/12 by Ramya Korlakai Vinayak, Vinayak, Ramya Korlakai, Weihao Kong +5 · 1 citation
Computer Science · Mathematics · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Statistical Methods and Bayesian Inference #Statistical Methods and Inference #Statistics Theory (math.ST)
- Robust Meta-learning for Mixed Linear Regression with Small Batches
2020/06/17 by Weihao Kong, Raghav Somani, Kong, Weihao +5 · 2 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Machine Learning and ELM
- Loss-to-Loss Prediction: Scaling Laws for All Datasets
2024/11/19 by David Brandfonbrener, Nikhil Anand, Brandfonbrener, David +7 · 3 citations
Computer Science · Mathematics · #Artificial Intelligence (cs.AI) #Bayesian Modeling and Causal Inference #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Healthcare #Statistical Methods and Inference
- Provable Representation Learning for Imitation Learning via Bi-level Optimization
2020/02/24 by Sanjeev Arora, Arora, Sanjeev, Simon S. Du +7 · 1 citation
Computer Science · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics
- Leverage Score Sampling for Faster Accelerated Regression and ERM
2017/11/22 by Naman Agarwal, Agarwal, Naman, Sham M. Kakade +9 · 2 citations
Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Matrix Theory and Algorithms
- How Important is the Train-Validation Split in Meta-Learning?
2020/10/12 by Yu Bai, Minshuo Chen, Bai, Yu +13 · 1 citation
Computer Science · Engineering · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Sparse and Compressive Sensing Techniques
- Instabilities of Offline RL with Pre-Trained Neural Representation
2021/03/08 by Ruosong Wang, Wang, Ruosong, Yifan Wu +5 · 1 citation
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Mathematics #Formal Methods in Verification #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Software Reliability and Analysis Research #Software Testing and Debugging Techniques
- Going Beyond Linear RL: Sample Efficient Neural Function Approximation
2021/07/14 by Baihe Huang, Kaixuan Huang, Huang, Baihe +11 · 1 citation
Computer Science · Engineering · #Reinforcement Learning in Robotics #Neural Networks and Reservoir Computing #Ferroelectric and Negative Capacitance Devices
- Optimal Gradient-based Algorithms for Non-concave Bandit Optimization
2021/07/09 by Baihe Huang, Kaixuan Huang, Huang, Baihe +11 · 1 citation
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Stochastic Gradient Optimization Techniques
- The Benefits of Implicit Regularization from SGD in Least Squares Problems
2021/08/10 by Difan Zou, Zou, Difan, Jingfeng Wu +9 · 1 citation
Computer Science · Engineering · #Stochastic Gradient Optimization Techniques #Sparse and Compressive Sensing Techniques #Machine Learning and Algorithms
- Efficient Algorithms for Large-scale Generalized Eigenvector Computation and Canonical Correlation Analysis
2016/04/13 by Rong Ge, Ge, Rong, Chi Jin +7 · 1 citation
Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #Topological and Geometric Data Analysis
- Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning
2023/06/14 by Nikhil Vyas, Vyas, Nikhil, Depen Morwani +9 · 1 citation
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Neural Networks and Applications
- Soup to go: mitigating forgetting during continual learning with model averaging
2025/01/09 by Anat Kleiman, Kleiman, Anat, Gintare Karolina Dziugaite +7 · 2 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Intelligent Tutoring Systems and Adaptive Learning #Machine Learning (cs.LG)
- Pareto Frontiers in Neural Feature Learning: Data, Compute, Width, and Luck
2023/09/07 by Benjamin Edelman, Surbhi Goel, Edelman, Benjamin L. +7 · 1 citation
Computer Science · #Domain Adaptation and Few-Shot Learning #Neural Networks and Applications #Machine Learning and Algorithms
- (weak) Calibration is Computationally Hard
2012/02/20 by Elad Hazan, Hazan, Elad, Sham M. Kakade +1 · 1 citation
Decision Sciences · Economics, Econometrics and Finance · #Auction Theory and Applications #Economic theories and models #Game Theory and Applications
- The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton
2025/10/10 by Natalie Abreu, Abreu, Natalie, Nikhil Vyas +5 · 3 citations
Physics and Astronomy · #Magnetic confinement fusion research
- Selective Underfitting in Diffusion Models
2025/10/01 by Ki‐Whan Song, J.M. Kim, Song, Kiwhan +9 · 5 citations
Computer Science · Physics and Astronomy · Mathematics · #Generative Adversarial Networks and Image Synthesis #Model Reduction and Neural Networks #Markov Chains and Monte Carlo Methods