Johannes von Oswald
- Transformers learn in-context by gradient descent
2022/12/15 by Johannes von Oswald, von Oswald, Johannes, Eyvind Niklasson +11 · 6 voices · 117 citations
Computer Science · #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning #Generative Adversarial Networks and Image Synthesis #cs.AI #cs.CL #cs.LG
- Uncovering mesa-optimization algorithms in Transformers
2023/09/11 by Johannes von Oswald, Maximilian Schlegel, von Oswald, Johannes +24 · 2 voices · 13 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Natural Language Processing Techniques #Topic Modeling #cs.AI #cs.LG
- Weight decay induces low-rank attention layers
2024/10/31 by Seijin Kobayashi, Yassir Akram, Kobayashi, Seijin +4 · 1 voice · 8 citations
Computer Science · Neuroscience · #FOS: Computer and information sciences #Functional Brain Connectivity Studies #Machine Learning (cs.LG) #Neural dynamics and brain function #Visual Attention and Saliency Detection #cs.LG
- Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning
2025/12/23 by Seijin Kobayashi, Kobayashi, Seijin, Yanick Schimpf +29 · 5 voices · 1 citation
Computer Science · #Reinforcement Learning in Robotics #Adversarial Robustness in Machine Learning #Neural Networks and Reservoir Computing
- Discovering modular solutions that generalize compositionally
2023/12/22 by Simon Schug, Schug, Simon, Seijin Kobayashi +15 · 4 citations
Computer Science · #Topic Modeling #Explainable Artificial Intelligence (XAI)
- MesaNet: Sequence Modeling by Locally Optimal Test-Time Training
2025/06/05 by Johannes von Oswald, von Oswald, Johannes, Nino Scherrer +30 · 11 citations
Computer Science · #Topic Modeling #Advanced Neural Network Applications #Domain Adaptation and Few-Shot Learning
- Learning where to learn: Gradient sparsity in meta and continual\n learning
2021/01/01 by Johannes von Oswald, Dominic Zhao, von Oswald, Johannes +11 · 2 citations
Computer Science · Physics and Astronomy · #Domain Adaptation and Few-Shot Learning #Human Pose and Action Recognition #Model Reduction and Neural Networks