Littwin, Etai

1. Distillation Scaling Laws
   2025/02/12 by Dan Busbridge, Busbridge, Dan, Amitis Shidani +9 · 9 voices · 15 citations
   Engineering · #Process Optimization and Integration
2. What Algorithms can Transformers Learn? A Study in Length Generalization
   2023/10/24 by Hattie Zhou, Zhou, Hattie, Arwen Bradley +13 · 2 voices · 21 citations
   Computer Science · #cs.LG #cs.AI #cs.CL #stat.ML
3. When can transformers reason with abstract symbols?
   2023/10/15 by Enric Boix-Adserà, Enric Boix-Adsera, Omid Saremi +12 · 1 voice · 4 citations
   Computer Science · #Advanced Text Analysis Techniques #Natural Language Processing Techniques #cs.AI #cs.CL #cs.LG
4. LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures
   2023/12/07 by Vimal Thilak, Thilak, Vimal, Chen Huang +13 · 9 citations
   Computer Science · #Face and Expression Recognition #Machine Learning and Data Classification #Anomaly Detection Techniques and Applications
5. The Slingshot Mechanism: An Empirical Study of Adaptive Optimizers and the Grokking Phenomenon
   2022/06/10 by Thilak, Vimal, Littwin, Etai, Zhai, Shuangfei +3 · 6 citations
   #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Optimization and Control (math.OC)
6. Transformers learn through gradual rank increase
   2023/06/12 by Boix-Adsera, Enric, Littwin, Etai, Abbe, Emmanuel +2 · 7 citations
   #FOS: Computer and information sciences #Machine Learning (cs.LG)
7. Tensor Programs IIb: Architectural Universality of Neural Tangent Kernel Training Dynamics
   2021/05/08 by Greg Yang, Yang, Greg, Etai Littwin +1 · 5 citations
   Computer Science · Mathematics · #Advanced Neural Network Applications #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Neural and Evolutionary Computing (cs.NE) #Probability (math.PR) #Stochastic Gradient Optimization Techniques #Tensor decomposition and applications
8. Vanishing Gradients in Reinforcement Finetuning of Language Models
   2023/10/31 by Noam Razin, Hattie Zhou, Razin, Noam +13 · 6 citations
   Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Natural Language Processing Techniques #Reinforcement Learning in Robotics #Topic Modeling
9. Tensor Programs IVb: Adaptive Optimization in the Infinite-Width Limit
   2023/08/03 by Greg Yang, Etai Littwin, Yang, Greg +1 · 5 citations
   Computer Science · #Parallel Computing and Optimization Techniques #Computational Physics and Python Applications #Stochastic Gradient Optimization Techniques
10. How JEPA Avoids Noisy Features: The Implicit Bias of Deep Linear Self Distillation Networks
    2024/07/03 by Etai Littwin, Littwin, Etai, Omid Saremi +12 · 1 voice · 5 citations
    Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #cs.LG
11. Enhancing JEPAs with Spatial Conditioning: Robust and Efficient Representation Learning
    2024/10/14 by Etai Littwin, Vimal Thilak, Littwin, Etai +3 · 1 voice
    Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #cs.CV #cs.LG
12. Learning Representation from Neural Fisher Kernel with Low-rank Approximation
    2022/02/04 by Ruixiang Zhang, Shuangfei Zhai, Zhang, Ruixiang +5 · 1 citation
    Computer Science · Mathematics · Physics and Astronomy · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Medical Image Segmentation Techniques #Model Reduction and Neural Networks #Tensor decomposition and applications
13. Adaptivity and Modularity for Efficient Generalization Over Task Complexity
    2023/10/13 by Samira Abnar, Abnar, Samira, Omid Saremi +19 · 1 citation
    Computer Science · #Machine Learning and Data Classification #Domain Adaptation and Few-Shot Learning #Multimodal Machine Learning Applications
14. UI-JEPA: Towards Active Perception of User Intent through Onscreen User Activity
    2024/09/06 by Yicheng Fu, Fu, Yicheng, Raviteja Anantha +6 · 1 citation
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #Data Visualization and Analytics #FOS: Computer and information sciences #Human-Computer Interaction (cs.HC) #Innovative Human-Technology Interaction #Machine Learning (cs.LG) #Virtual Reality Applications and Impacts