Cordonnier, Jean-Baptiste

1. Sparsified SGD with Memory
   2018/09/20 by Sebastian U. Stich, Jean-Baptiste Cordonnier, Stich, Sebastian U. +3 · 48 citations
   Computer Science · Engineering · #68W15 #68W40 #90C06 #90C25 #Data Structures and Algorithms (cs.DS) #Distributed #Distributed Sensor Networks and Detection Algorithms #E.4 #F.2.1 #FOS: Computer and information sciences #G.1.6 #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Parallel #Sparse and Compressive Sensing Techniques #Stochastic Gradient Optimization Techniques #and Cluster Computing (cs.DC)
2. Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth
   2021/03/04 by Yihe Dong, Dong, Yihe, Jean-Baptiste Cordonnier +3 · 36 citations
   Computer Science · Engineering · #Advanced Memory and Neural Computing #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Neural Networks and Reservoir Computing
3. On the Relationship between Self-Attention and Convolutional Layers
   2019/11/08 by Cordonnier, Jean-Baptiste, Loukas, Andreas, Jaggi, Martin · 14 citations
   #Computation and Language (cs.CL) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)
4. Multi-Head Attention: Collaborate Instead of Concatenate
   2020/06/29 by Cordonnier, Jean-Baptiste, Loukas, Andreas, Jaggi, Martin · 7 citations
   #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)
5. Differentiable Patch Selection for Image Recognition
   2021/04/07 by Cordonnier, Jean-Baptiste, Mahendran, Aravindh, Dosovitskiy, Alexey +3 · 3 citations
   #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML)