Weigao Sun
- MiniMax-01: Scaling Foundation Models with Lightning Attention
2025/01/14 by MiniMax, Aonian Li, Ao Li +191 · 3 voices · 36 citations
Computer Science · #Computation and Language (cs.CL) #Computer Graphics and Visualization Techniques #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #cs.CL #cs.CV
- HGRN2: Gated Linear RNNs with State Expansion
2024/04/11 by Zhen Qin, Songlin Yang, Qin, Zhen +11 · 19 citations
Computer Science · #Advanced Neural Network Applications #Computation and Language (cs.CL) #FOS: Computer and information sciences #Speech Recognition and Synthesis #Topic Modeling
- A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
2025/03/27 by Xiaoye Qu, Qu, Xiaoye, Yafu Li +31 · 39 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Semantic Web and Ontologies #Speech and dialogue systems
- Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models
2024/01/09 by Zhen Qin, Qin, Zhen, Weigao Sun +9 · 9 citations
Computer Science · #Advanced Neural Network Applications #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Handwritten Text Recognition Techniques #Topic Modeling
- TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer
2023/07/27 by Zhen Qin, Qin, Zhen, Li Dong +19 · 9 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Natural Language Processing Techniques #Topic Modeling
- MoM: Linear Sequence Modeling with Mixture-of-Memories
2025/02/19 by Jusen Du, Du, Jusen, Weigao Sun +7 · 1 voice · 9 citations
#cs.CL #cs.AI #cs.LG
- Scaling Laws for Linear Complexity Language Models
2024/06/24 by Xuyang Shen, Shen, Xuyang, Dong Li +9 · 5 citations
Computer Science · #Natural Language Processing Techniques
- LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Training
2024/11/24 by Xiaoye Qu, Qu, Xiaoye, Daize Dong +9 · 5 citations
Computer Science · #Machine Learning and Algorithms #Computability, Logic, AI Algorithms #Topic Modeling
- Liger: Linearizing Large Language Models to Gated Recurrent Structures
2025/03/03 by Disen Lan, Lan, Disen, Weigao Sun +7 · 5 citations
Computer Science · #Natural Language Processing Techniques #Topic Modeling
- CO2: Efficient Distributed Training with Full Communication-Computation Overlap
2024/01/29 by Weigao Sun, Zhen Qin, Sun, Weigao +13 · 2 citations
Medicine · Computer Science · #Medical Imaging Techniques and Applications #IoT and Edge/Fog Computing #Energy Efficient Wireless Sensor Networks
- Comba: Improving Bilinear RNNs with Closed-loop Control
2025/06/03 by Jiaxi Hu, Hu, Jiaxi, Yongqi Pan +12 · 2 citations
Computer Science · #Topic Modeling #Neural Networks and Applications #Adversarial Robustness in Machine Learning
- LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid
2025/02/11 by Weigao Sun, Disen Lan, Sun, Weigao +7 · 2 citations
Computer Science · #Anomaly Detection Techniques and Applications #Artificial Intelligence (cs.AI) #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Neural Networks and Applications #Parallel Computing and Optimization Techniques
- Unlocking the Secrets of Linear Complexity Sequence Model from A Unified Perspective
2024/05/27 by Zhen Qin, Qin, Zhen, Xuyang Shen +11 · 1 citation
Computer Science · #Computation and Language (cs.CL) #Data Mining and Machine Learning Applications #FOS: Computer and information sciences