- Uncovering mesa-optimization algorithms in Transformers
2023/09/11 by Johannes von Oswald, von Oswald, Johannes, Maximilian Schlegel +24 · 2 voices · 26 citations
Computer Science · Engineering · Mathematics · #Algorithm #Artificial intelligence #Autoregressive model #Computer science #Domain Adaptation and Few-Shot Learning #Electrical engineering #Engineering #Language model #Machine learning #Mathematical optimization #Mathematics #Mesa #Natural Language Processing Techniques #Optimization algorithm #Optimization problem #Topic Modeling #Transformer #Voltage #cs.AI #cs.LG