vix.ing · top · new · best · stats · spec

Noam Shazeer

  1. Attention Is All You Need
    The Transformer, an attention-only architecture, beat prior best translation results while training far faster than RNN or CNN models.
    2017/06/12 by Ashish Vaswani, Noam Shazeer, Vaswani, Ashish +13 · 85 voices · 6573 citations
    #cs.CL #cs.LG
  2. Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
    Sparsely-Gated MoE layers increase model capacity by over 1000x with minimal compute loss, improving language modeling and translation.
    2017/01/23 by Noam Shazeer, Azalia Mirhoseini, Shazeer, Noam +13 · 11 voices · 657 citations
    Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Topic Modeling #cs.CL #cs.LG #cs.NE #stat.ML
  3. One Model To Learn Them All
    2017/06/16 by Lukasz Kaiser, Łukasz Kaiser, Aidan N. Gomez +12 · 11 voices · 10 citations
    Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.LG #stat.ML
  4. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
    2021/01/11 by William Fedus, Barret Zoph, Fedus, William +3 · 9 voices · 487 citations
    Computer Science · #Domain Adaptation and Few-Shot Learning #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.LG
  5. GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
    2020/06/30 by Dmitry Lepikhin, HyoukJoong Lee, Lepikhin, Dmitry +16 · 2 voices · 248 citations
    Computer Science · #Advanced Neural Network Applications #Machine Learning and Data Classification #Topic Modeling #cs.CL #cs.LG #stat.ML
  6. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
    2019/10/23 by Colin Raffel, Raffel, Colin, Noam Shazeer +16 · 4 voices · 1172 citations
    Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling #cs.CL #cs.LG #stat.ML
  7. Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
    2025/07/07 by Gheorghe Comanici, Comanici, Gheorghe, Eric Bieber +6844 · 8 voices · 1350 citations
    #cs.CL #cs.AI
  8. GLU Variants Improve Transformer
    2020/02/12 by Noam Shazeer, Shazeer, Noam · 6 voices · 242 citations
    Computer Science · Engineering · #Analog and Mixed-Signal Circuit Design #Neural Networks and Reservoir Computing #Parallel Computing and Optimization Techniques #cs.LG #cs.NE #stat.ML
  9. Exploring the Limits of Language Modeling
    2016/02/07 by Rafal Jozefowicz, Rafał Józefowicz, Oriol Vinyals +8 · 2 voices · 52 citations
    Computer Science · #Natural Language Processing Techniques #Speech and dialogue systems #Topic Modeling #cs.CL
  10. Fast Transformer Decoding: One Write-Head is All You Need
    2019/11/06 by Noam Shazeer, Shazeer, Noam · 1 voice · 86 citations
    Computer Science · #cs.NE #cs.CL #cs.LG
  11. PaLM: Scaling Language Modeling with Pathways
    2022/04/05 by Aakanksha Chowdhery, Sharan Narang, Chowdhery, Aakanksha +129 · 391 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  12. LaMDA: Language Models for Dialog Applications
    2022/01/20 by Romal Thoppilan, Daniel De Freitas, Thoppilan, Romal +117 · 2 voices · 46 citations
    #cs.CL #cs.AI
  13. ST-MoE: Designing Stable and Transferable Sparse Expert Models
    2022/02/17 by Barret Zoph, Irwan Bello, Zoph, Barret +13 · 1 voice · 47 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Multimodal Machine Learning Applications
  14. Gemma 3 Technical Report
    2025/03/25 by Aishwarya Kamath, Gemma Team, Johan Ferret +418 · 3 voices · 462 citations
    Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #cs.AI #cs.CL
  15. Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks
    2015/06/09 by Samy Bengio, Bengio, Samy, Oriol Vinyals +5 · 134 citations
    Computer Science · #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Topic Modeling
  16. Generating Wikipedia by Summarizing Long Sequences
    2018/01/30 by Peter J. Liu, Liu, Peter J., Mohammad Saleh +12 · 1 voice · 25 citations
    Computer Science · Social Sciences · #Natural Language Processing Techniques #Topic Modeling #Wikis in Education and Collaboration #cs.CL
  17. Adafactor: Adaptive Learning Rates with Sublinear Memory Cost
    2018/04/11 by Noam Shazeer, Shazeer, Noam, Mitchell Stern +1 · 80 citations
    Computer Science · Physics and Astronomy · #Stochastic Gradient Optimization Techniques #Neural Networks and Applications #Model Reduction and Neural Networks
  18. Music Transformer
    2018/09/12 by Cheng-Zhi Anna Huang, Ashish Vaswani, Huang, Cheng-Zhi Anna +17 · 19 citations
    Computer Science · Neuroscience · #Music and Audio Processing #Music Technology and Sound Studies #Neuroscience and Music Perception
  19. Primer: Searching for Efficient Transformers for Language Modeling
    2021/09/17 by David R. So, Wojciech Mańke, So, David R. +9 · 10 citations
    Computer Science · #Topic Modeling #Natural Language Processing Techniques #Speech Recognition and Synthesis
  20. End-to-End Text-Dependent Speaker Verification
    2015/09/27 by Georg Heigold, Ignacio López Moreno, Heigold, Georg +5 · 5 citations
    Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Sound (cs.SD) #Speech Recognition and Synthesis #Topic Modeling
  21. Swivel: Improving Embeddings by Noticing What's Missing
    2016/02/06 by Noam Shazeer, Ryan Doherty, Shazeer, Noam +5 · 1 voice · 1 citation
    Computer Science · #Advanced Graph Neural Networks #Domain Adaptation and Few-Shot Learning #Face and Expression Recognition #cs.CL
  22. Fast Decoding in Sequence Models using Discrete Latent Variables
    2018/03/09 by Łukasz Kaiser, Kaiser, Łukasz, Aurko Roy +11 · 9 citations
    Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis #Topic Modeling
  23. Talking-Heads Attention
    2020/03/05 by Noam Shazeer, Shazeer, Noam, Zhenzhong Lan +7 · 3 citations
    Computer Science · #Audio and Speech Processing (eess.AS) #FOS: Computer and information sciences #FOS: Electrical engineering #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Multimodal Machine Learning Applications #Natural Language Processing Techniques #Neural and Evolutionary Computing (cs.NE) #Sound (cs.SD) #Topic Modeling #electronic engineering #information engineering