Tengyu Ma
- On the Opportunities and Risks of Foundation Models
2021/08/16 by Rishi Bommasani, Drew A. Hudson, Bommasani, Rishi +233 · 11 voices · 547 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Adversarial Robustness in Machine Learning #Topic Modeling
- Sophia: A Scalable Stochastic Second-order Optimizer for Language Model Pre-training
2023/05/23 by Hong Liu, Liu, Hong, Zhiyuan Li +8 · 7 voices · 42 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Advanced Neural Network Applications
- Chain of Thought Empowers Transformers to Solve Inherently Serial Problems
2024/02/20 by Zhiyuan Li, Li, Zhiyuan, Hong Liu +5 · 8 voices · 63 citations
#cs.LG #cs.CC #stat.ML
- What learning algorithm is in-context learning? Investigations with linear models
2022/11/28 by Ekin Akyürek, Dale Schuurmans, Akyürek, Ekin +7 · 3 voices · 72 citations
Computer Science · #Neural Networks and Applications #cs.CL #cs.LG
- SAM 2: Segment Anything in Images and Videos
2024/08/01 by Nikhila Ravi, Ravi, Nikhila, Valentin Gabeur +33 · 690 citations
Computer Science · #Generative Adversarial Networks and Image Synthesis
- Large Language Models as Tool Makers
2023/05/26 by Tianle Cai, Xuezhi Wang, Cai, Tianle +7 · 2 voices · 28 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.CL #cs.LG #stat.ML
- An Explanation of In-context Learning as Implicit Bayesian Inference
2021/11/03 by Sang Michael Xie, Xie, Sang Michael, Aditi Raghunathan +5 · 91 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
- Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss
2019/06/18 by Kaidi Cao, Cao, Kaidi, Colin Wei +7 · 63 citations
Computer Science · #Anomaly Detection Techniques and Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Imbalanced Data Classification Techniques #Machine Learning (cs.LG) #Machine Learning (stat.ML)
- Fantastic Pretraining Optimizers and Where to Find Them
2025/09/02 by Kaiyue Wen, Wen, Kaiyue, David Hall +5 · 5 voices · 26 citations
#cs.LG #cs.AI #stat.ML
- Perception Encoder: The best visual embeddings are not at the output of the network
2025/04/17 by Daniel Bolya, Bolya, Daniel, Po-Yao Huang +35 · 2 voices · 104 citations
Computer Science · #Multimodal Machine Learning Applications #Generative Adversarial Networks and Image Synthesis #Human Pose and Action Recognition
- DoReMi: Optimizing Data Mixtures Speeds Up Language Model Pretraining
2023/05/17 by Sang Michael Xie, Hieu Pham, Xie, Sang Michael +17 · 58 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
- Data Selection for Language Models via Importance Resampling
2023/02/06 by Sang Michael Xie, Shibani Santurkar, Xie, Sang Michael +5 · 50 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling #Speech Recognition and Synthesis
- MOPO: Model-based Offline Policy Optimization
2020/05/27 by Tianhe Yu, Yu, Tianhe, Garrett Thomas +13 · 37 citations
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Formal Methods in Verification #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Reinforcement Learning in Robotics #Simulation Techniques and Applications
- Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss
2021/06/08 by Jeff Z. HaoChen, Colin Wei, HaoChen, Jeff Z. +5 · 2 voices · 23 citations
#cs.LG #stat.ML
- One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention
2023/07/07 by Arvind V. Mahankali, Mahankali, Arvind, Tatsunori Hashimoto +3 · 31 citations
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification
- Larger language models do in-context learning differently
2023/03/07 by Jerry Wei, Wei, Jerry, Wei, Jason +18 · 25 citations
Computer Science · #Computation and Language (cs.CL) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling
- STP: Self-play LLM Theorem Provers with Iterative Conjecturing and Proving
2025/01/31 by Kefan Dong, Dong, Kefan, Tengyu Ma +1 · 2 voices · 14 citations
Computer Science · Decision Sciences · #Artificial Intelligence (cs.AI) #Auction Theory and Applications #FOS: Computer and information sciences #Logic in Computer Science (cs.LO) #Machine Learning (cs.LG) #cs.AI #cs.LG #cs.LO
- Learning One-hidden-layer Neural Networks with Landscape Design
2017/11/01 by Rong Ge, Jason D. Lee, Ge, Rong +3 · 21 citations
Computer Science · #Stochastic Gradient Optimization Techniques #Machine Learning and Algorithms #Machine Learning and ELM
- Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel
2018/10/12 by Colin Wei, Jason D. Lee, Wei, Colin +5 · 14 citations
Computer Science · #Advanced Neural Network Applications #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Stochastic Gradient Optimization Techniques
- Identity Matters in Deep Learning
2016/11/14 by Moritz Hardt, Hardt, Moritz, Tengyu Ma +1 · 11 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE)
- Shape Matters: Understanding the Implicit Bias of the Noise Covariance
2020/06/15 by Jeff Z. HaoChen, Colin Wei, HaoChen, Jeff Z. +5 · 14 citations
Computer Science · #Machine Learning and Algorithms #Stochastic Gradient Optimization Techniques #Adversarial Robustness in Machine Learning
- Data-dependent Sample Complexity of Deep Neural Networks via Lipschitz Augmentation
2019/05/09 by Colin Wei, Wei, Colin, Tengyu Ma +1 · 12 citations
Engineering · Computer Science · #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #Machine Learning and ELM
- Calibrating Predictions to Decisions: A Novel Approach to Multi-Class Calibration
2021/07/12 by Shengjia Zhao, Zhao, Shengjia, Michael P. Kim +8 · 9 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #Adversarial Robustness in Machine Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Imbalanced Data Classification Techniques #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Methodology (stat.ME)
- Communication Lower Bounds for Statistical Estimation Problems via a\n Distributed Data Processing Inequality
2015/06/23 by Mark Braverman, Ankit Garg, Braverman, Mark +7 · 6 citations
Mathematics · Computer Science · #Statistical Methods and Inference #Stochastic Gradient Optimization Techniques #Privacy-Preserving Technologies in Data
- Optimal Regularization Can Mitigate Double Descent
2020/03/04 by Preetum Nakkiran, Prayaag Venkat, Nakkiran, Preetum +5 · 8 citations
Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE) #Sparse and Compressive Sensing Techniques #Statistics Theory (math.ST) #Stochastic Gradient Optimization Techniques
- The Implicit and Explicit Regularization Effects of Dropout
2020/02/28 by Colin Wei, Wei, Colin, Sham M. Kakade +3 · 7 citations
Computer Science · #Adversarial Robustness in Machine Learning #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Stochastic Gradient Optimization Techniques
- SAM 3: Segment Anything with Concepts
2025/11/20 by Nicolas Carion, Carion, Nicolas, Laura Gustafson +73 · 49 citations
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Multimodal Machine Learning Applications
- Theoretical Analysis of Self-Training with Deep Networks on Unlabeled Data
2020/10/07 by Colin Wei, Kendrick Shen, Wei, Colin +5 · 7 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification #Machine Learning and Algorithms
- On Communication Cost of Distributed Statistical Estimation and\n Dimensionality
2014/05/07 by Ankit Garg, Garg, Ankit, Tengyu Ma +3 · 5 citations
Computer Science · #Distributed Sensor Networks and Detection Algorithms #FOS: Computer and information sciences #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning and Algorithms #Privacy-Preserving Technologies in Data #Stochastic Gradient Optimization Techniques
- Label Noise SGD Provably Prefers Flat Global Minimizers
2021/06/11 by Alex Damian, Tengyu Ma, Damian, Alex +3 · 10 citations
Computer Science · #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #FOS: Mathematics #Information Theory (cs.IT) #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Machine Learning and Data Classification #Optimization and Control (math.OC)
- PerceptionLM: Open-Access Data and Models for Detailed Visual Understanding
2025/04/17 by Jang Hyun Cho, Andrea Madotto, Cho, Jang Hyun +54 · 30 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Multimodal Machine Learning Applications
- Linguistic Calibration of Long-Form Generations
2024/03/30 by Neil Band, Xuechen Li, Band, Neil +5 · 12 citations
Computer Science · #Natural Language Processing Techniques
- Symbol tuning improves in-context learning in language models
2023/05/15 by Jerry Wei, Wei, Jerry, Le Hou +19 · 9 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Text Readability and Simplification #Topic Modeling
- How Does Sharpness-Aware Minimization Minimize Sharpness?
2022/11/10 by Kaiyue Wen, Tengyu Ma, Wen, Kaiyue +3 · 8 citations
Computer Science · #Stochastic Gradient Optimization Techniques #Domain Adaptation and Few-Shot Learning #Machine Learning and ELM
- Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization
2023/07/20 by Kaiyue Wen, Wen, Kaiyue, Zhiyuan Li +3 · 9 citations
Computer Science · #Neural Networks and Applications #Stochastic Gradient Optimization Techniques #Machine Learning and ELM
- Understanding Self-Training for Gradual Domain Adaptation
2020/02/26 by Ananya Kumar, Tengyu Ma, Kumar, Ananya +3 · 6 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Machine Learning and ELM #Stochastic Gradient Optimization Techniques
- Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling
2020/10/21 by Wenxuan Zhou, Zhou, Wenxuan, Kevin Huang +5 · 6 citations
Biochemistry, Genetics and Molecular Biology · Computer Science · #Biomedical Text Mining and Ontologies #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling
- Same Pre-training Loss, Better Downstream: Implicit Bias Matters for Language Models
2022/10/25 by Hong Liu, Sang Michael Xie, Liu, Hong +5 · 7 citations
Computer Science · #Topic Modeling #Natural Language Processing Techniques #Speech Recognition and Synthesis
- Self-supervised Learning is More Robust to Dataset Imbalance
2021/10/11 by Hong Liu, Jeff Z. HaoChen, Liu, Hong +5 · 6 citations
Computer Science · Engineering · #Domain Adaptation and Few-Shot Learning #Imbalanced Data Classification Techniques #Infrastructure Maintenance and Monitoring
- Individual Calibration with Randomized Forecasting
2020/06/18 by Shengjia Zhao, Zhao, Shengjia, Tengyu Ma +3 · 5 citations
Computer Science · Decision Sciences · #Adversarial Robustness in Machine Learning #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Forecasting Techniques and Applications #Machine Learning (cs.LG) #Machine Learning (stat.ML)
- Fine-Grained Gap-Dependent Bounds for Tabular MDPs via Adaptive Multi-Step Bootstrap
2021/02/09 by Haike Xu, Tengyu Ma, Xu, Haike +3 · 7 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Reinforcement Learning in Robotics
- Distributed Stochastic Variance Reduced Gradient Methods and A Lower Bound for Communication Complexity
2015/07/27 by Jason D. Lee, Qihang Lin, Lee, Jason D. +5 · 7 citations
Computer Science · Engineering · #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Optimization and Control (math.OC) #Privacy-Preserving Technologies in Data #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques
- Plan Better Amid Conservatism: Offline Multi-Agent Reinforcement Learning with Actor Rectification
2021/11/22 by Ling Pan, Longbo Huang, Pan, Ling +5 · 6 citations
Computer Science · Engineering · #Adaptive Dynamic Programming Control #Elevator Systems and Control #FOS: Computer and information sciences #Machine Learning (cs.LG) #Reinforcement Learning in Robotics
- A Theoretical Study of Inductive Biases in Contrastive Learning
2022/11/27 by Jeff Z. HaoChen, HaoChen, Jeff Z., Tengyu Ma +1 · 5 citations
Computer Science · #Domain Adaptation and Few-Shot Learning
- Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers
2021/07/28 by Colin Wei, Yining Chen, Wei, Colin +3 · 5 citations
Computer Science · #Neural Networks and Applications #Machine Learning and Algorithms #Stochastic Gradient Optimization Techniques
- Why Do Pretrained Language Models Help in Downstream Tasks? An Analysis of Head and Prompt Tuning
2021/06/17 by Colin Wei, Sang Michael Xie, Wei, Colin +3 · 5 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
- DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization
2021/12/09 by Aviral Kumar, Kumar, Aviral, Rishabh Agarwal +9 · 5 citations
Computer Science · #Reinforcement Learning in Robotics #Domain Adaptation and Few-Shot Learning #Machine Learning and ELM
- Improved Sample Complexities for Deep Networks and Robust Classification via an All-Layer Margin
2019/10/09 by Colin Wei, Wei, Colin, Tengyu Ma +1 · 6 citations
Computer Science · #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms
- Self-training Avoids Using Spurious Features Under Domain Shift
2020/06/17 by Yining Chen, Chen, Yining, Colin Wei +5 · 3 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and ELM #Multimodal Machine Learning Applications
- Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective
2024/10/07 by Kaiyue Wen, Wen, Kaiyue, Zhiyuan Li +9 · 8 citations
Computer Science · #Machine Learning and Data Classification #Machine Learning and Algorithms #Anomaly Detection Techniques and Applications
- Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation
2022/04/01 by Kendrick Shen, Shen, Kendrick, R. Jones +11 · 3 citations
Computer Science · Medicine · #COVID-19 diagnosis using AI #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Multimodal Machine Learning Applications
- On the Performance of Thompson Sampling on Logistic Bandits
2019/05/12 by Tengyu Ma, Dong, Shi, Benjamin Van Roy +2 · 2 citations
Computer Science · Decision Sciences · #Advanced Bandit Algorithms Research #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Algorithms #Reinforcement Learning in Robotics
- Provable Model-based Nonlinear Bandit and Reinforcement Learning: Shelve\n Optimism, Embrace Virtual Curvature
2021/02/08 by Kefan Dong, Dong, Kefan, Jiaqi Yang +3 · 4 citations
Decision Sciences · Computer Science · #Advanced Bandit Algorithms Research #Adaptive Dynamic Programming Control #Reinforcement Learning in Robotics
- Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violations
2021/08/04 by Yuping Luo, Luo, Yuping, Tengyu Ma +1 · 2 citations
Computer Science · Engineering · #Adversarial Robustness in Machine Learning #Artificial Intelligence (cs.AI) #Autonomous Vehicle Technology and Safety #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Reinforcement Learning in Robotics #Robotics (cs.RO) #Systems and Control (eess.SY) #electronic engineering #information engineering
- First Steps Toward Understanding the Extrapolation of Nonlinear Models to Unseen Domains
2022/11/21 by Kefan Dong, Dong, Kefan, Tengyu Ma +1 · 1 citation
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning in Healthcare #Neural Networks and Applications
- Trash to Treasure: Low-Light Object Detection via Decomposition-and-Aggregation
2023/09/07 by Xiaohan Cui, Long Ma, Cui, Xiaohan +9 · 1 citation
Computer Science · #Advanced Image and Video Retrieval Techniques #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Visual Attention and Saliency Detection
- In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness
2020/12/08 by Sang Michael Xie, Ananya Kumar, Xie, Sang Michael +9 · 2 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Multimodal Machine Learning Applications