Saremi, Omid
- What Algorithms can Transformers Learn? A Study in Length Generalization
2023/10/24 by Hattie Zhou, Zhou, Hattie, Arwen Bradley +13 · 2 voices · 35 citations
Computer Science · Mathematics · #cs.LG #cs.AI #cs.CL #stat.ML
- How Far Can Transformers Reason? The Globality Barrier and Inductive Scratchpad
2024/06/10 by Emmanuel Abbé, Emmanuel Abbe, Samy Bengio +8 · 1 voice · 12 citations
Computer Science · Mathematics · Psychology · #Computability, Logic, AI Algorithms #Deception detection and forensic psychology #Neural Networks and Applications #cs.AI #cs.LG #stat.ML
- When can transformers reason with abstract symbols?
2023/10/15 by Enric Boix-Adsera, Enric Boix-Adserà, Omid Saremi +12 · 1 voice · 8 citations
Computer Science · #Advanced Text Analysis Techniques #Natural Language Processing Techniques #cs.AI #cs.CL #cs.LG
- The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon
2022/06/10 by Vimal Thilak, Thilak, Vimal, Etai Littwin +9 · 10 citations
Computer Science · Physics and Astronomy · #Advanced Thermodynamics and Statistical Mechanics #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC) #Quantum Information and Cryptography #Quantum many-body systems
- LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures
2023/12/07 by Vimal Thilak, Chen Huang, Thilak, Vimal +13 · 12 citations
Computer Science · #Face and Expression Recognition #Machine Learning and Data Classification #Anomaly Detection Techniques and Applications
- Vanishing Gradients in Reinforcement Finetuning of Language Models
2023/10/31 by Noam Razin, Razin, Noam, Hattie Zhou +13 · 10 citations
Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques #Reinforcement Learning in Robotics #Topic Modeling
- Adaptivity and Modularity for Efficient Generalization Over Task Complexity
2023/10/13 by Samira Abnar, Omid Saremi, Abnar, Samira +20 · 1 voice · 2 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification #Multimodal Machine Learning Applications #cs.AI #cs.LG
- How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks
2024/07/03 by Etai Littwin, Omid Saremi, Littwin, Etai +12 · 1 voice · 7 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #cs.LG
- Disorder in Gauge/Gravity Duality, Pole Spectrum Statistics and Random Matrix Theory
2012/06/08 by Omid Saremi, Saremi, Omid · 1 citation
Physics and Astronomy · #Disordered Systems and Neural Networks (cond-mat.dis-nn) #FOS: Physical sciences #General Relativity and Quantum Cosmology (gr-qc) #High Energy Physics - Theory (hep-th) #cond-mat.dis-nn #gr-qc #hep-th
- Shear Waves, Sound Waves On A Shimmering Horizon
2007/03/19 by Omid Saremi, Saremi, Omid · 1 citation
Earth and Planetary Sciences · Physics and Astronomy · #Black Holes and Theoretical Physics #Cosmology and Gravitation Theories #FOS: Physical sciences #High Energy Physics - Theory (hep-th) #Seismic Waves and Analysis #hep-th