Sheen, Heejune

1. Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality
   2024/02/29 by Siyu Chen, Heejune Sheen, Chen, Siyu +5 · 19 citations
   Computer Science · #Neural Networks and Applications
2. Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers
   2024/09/09 by Siyu Chen, Chen, Siyu, Heejune Sheen +5 · 18 citations
   Computer Science · #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #FOS: Mathematics #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Optimization and Control (math.OC)
3. Implicit Regularization of Gradient Flow on One-Layer Softmax Attention
   2024/03/13 by Heejune Sheen, Siyu Chen, Sheen, Heejune +5 · 2 citations
   Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #FOS: Mathematics #Image and Signal Denoising Methods #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Neural Networks and Applications #Optimization and Control (math.OC)