Roman Novak
- Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
2022/06/09 by Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao +448 · 3 voices · 126 citations
#cs.CL #cs.AI #cs.CY #cs.LG #stat.ML
- Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent
2019/02/18 by Jaehoon Lee, Lechao Xiao, Samuel S. Schoenholz +4 · 2 voices · 11 citations
Mathematics · Computer Science · #stat.ML #cs.LG
- Deep Neural Networks as Gaussian Processes
2017/11/01 by Jaehoon Lee, Yasaman Bahri, Lee, Jaehoon +9 · 59 citations
Computer Science · Physics and Astronomy · #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Statistical Mechanics and Entropy #Target Tracking and Data Fusion in Sensor Networks
- Scaling Exponents Across Parameterizations and Optimizers
2024/07/08 by Katie Everett, Lechao Xiao, Everett, Katie +19 · 2 voices · 18 citations
#cs.LG
- Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models
2023/12/11 by Avi Singh, John D. Co-Reyes, Singh, Avi +76 · 35 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification #Natural Language Processing Techniques #Topic Modeling
- Sensitivity and Generalization in Neural Networks: an Empirical Study
2018/02/23 by Roman Novak, Yasaman Bahri, Novak, Roman +7 · 20 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Neural and Evolutionary Computing (cs.NE)
- Small-scale proxies for large-scale Transformer training instabilities
2023/09/25 by Mitchell Wortsman, Wortsman, Mitchell, Peter J. Liu +29 · 25 citations
Computer Science · Decision Sciences · #Explainable Artificial Intelligence (XAI) #FOS: Computer and information sciences #Forecasting Techniques and Applications #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG)
- Bayesian Deep Convolutional Networks with Many Channels are Gaussian\n Processes
2018/10/11 by Roman Novak, Novak, Roman, Lechao Xiao +16 · 21 citations
Computer Science · #Adversarial Robustness in Machine Learning #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Machine Learning and Data Classification #Neural and Evolutionary Computing (cs.NE)
- Neural Tangents: Fast and Easy Infinite Neural Networks in Python
2019/12/05 by Roman Novak, Novak, Roman, Lechao Xiao +11 · 11 citations
Physics and Astronomy · Computer Science · #Model Reduction and Neural Networks #Advanced Neural Network Applications #Adversarial Robustness in Machine Learning
- Fast Finite Width Neural Tangent Kernel
2022/06/17 by Roman Novak, Novak, Roman, Jascha Sohl‐Dickstein +3 · 8 citations
Physics and Astronomy · Computer Science · #Model Reduction and Neural Networks #Neural Networks and Applications #Human Pose and Action Recognition
- On the infinite width limit of neural networks with a standard\n parameterization
2020/01/20 by Jascha Sohl‐Dickstein, Roman Novak, Sohl-Dickstein, Jascha +5 · 4 citations
Computer Science · Physics and Astronomy · #Neural Networks and Applications #Advanced Neural Network Applications #Model Reduction and Neural Networks