Shiwei Liu
- The Curse of Depth in Large Language Models
2025/02/09 by Wenfang Sun, Xinyuan Song, Sun, Wenfang +9 · 6 voices · 10 citations
Computer Science · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques
- Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
2023/10/08 by Lu Yin, You Wu, Yin, Lu +24 · 1 voice · 24 citations
Computer Science · #cs.LG
- AdaMerging: Adaptive Model Merging for Multi-Task Learning
2023/10/04 by Enneng Yang, Yang, Enneng, Zhenyi Wang +11 · 51 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #Machine Learning and Data Classification #Multimodal Machine Learning Applications
- Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding
2024/03/05 by Zhenyu Zhang, Runjin Chen, Zhang, Zhenyu +13 · 19 citations
Computer Science · #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Speech and dialogue systems
- More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity
2022/07/07 by Shiwei Liu, Liu, Shiwei, Tianlong Chen +15 · 11 citations
Computer Science · Medicine · #Advanced Neural Network Applications #COVID-19 diagnosis using AI #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences
- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs
2023/10/13 by Yuxin Zhang, Lirui Zhao, Zhang, Yuxin +15 · 12 citations
Computer Science · #Artificial Intelligence (cs.AI) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Speech Recognition and Synthesis #Topic Modeling
- Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers
2023/03/02 by Tianlong Chen, Zhenyu Zhang, Chen, Tianlong +7 · 7 citations
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and ELM #Multimodal Machine Learning Applications
- Suicide risk and prevention during the COVID-19 pandemic
2020/04/21 by Louis Appleby, Ella Arensman, Lai Fong Chan +43 · 7 citations
Medicine · Psychology · #COVID-19 and Mental Health #Long-Term Effects of COVID-19 #Suicide and Self-Harm Studies
- Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning
2024/10/09 by Abhinav Bandari, Lu Yin, Bandari, Abhinav +13 · 6 citations
Engineering · #Advanced Surface Polishing Techniques #Computation and Language (cs.CL) #FOS: Computer and information sciences #Metallurgical Processes and Thermodynamics #Mineral Processing and Grinding
- AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models
2024/10/14 by Haiquan Lu, Yefan Zhou, Lu, Haiquan +8 · 7 citations
Computer Science · #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning (stat.ML) #Topic Modeling
- E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation
2023/12/07 by Boqian Wu, Wu, Boqian, Xiao, Qiao +12 · 3 citations
Computer Science · Medicine · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Medical Image Segmentation Techniques #Radiomics and Machine Learning in Medical Imaging
- Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion
2024/06/14 by Anke Tang, Tang, Anke, Li Shen +9 · 3 citations
Computer Science · Engineering · #Artificial Intelligence (cs.AI) #FOS: Computer and information sciences #Gaussian Processes and Bayesian Inference #Machine Learning (cs.LG) #Neural Networks and Applications #Reservoir Engineering and Simulation Methods
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients
2024/07/11 by Zhenyu Zhang, Ajay Jaiswal, Zhang, Zhenyu +11 · 3 citations
Engineering · Materials Science · Physics and Astronomy · #FOS: Computer and information sciences #Ga2O3 and related materials #GaN-based semiconductor devices and materials #Machine Learning (cs.LG) #Semiconductor materials and devices
- On improving deep learning generalization with adaptive sparse connectivity
2019/06/27 by Shiwei Liu, Decebal Constantin Mocanu, Liu, Shiwei +3 · 1 citation
Computer Science · #Evolutionary Algorithms and Applications #Machine Learning and ELM #Metaheuristic Optimization Algorithms Research
- Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective
2023/12/03 by Can Jin, Tianjin Huang, Jin, Can +11 · 2 citations
Computer Science · #Advanced Neural Network Applications #Computer Vision and Pattern Recognition (cs.CV) #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Machine Learning and Data Classification
- From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications
2024/07/15 by Ajay Jaiswal, Jaiswal, Ajay, Wang, Yifan +12 · 2 citations
Materials Science · Engineering · #Ga2O3 and related materials #Plasma Diagnostics and Applications #High voltage insulation and dielectric phenomena
- Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance
2022/03/05 by Shiwei Liu, Liu, Shiwei, Yuesong Tian +5 · 1 citation
Computer Science · #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #Generative Adversarial Networks and Image Synthesis #Machine Learning (cs.LG) #Music and Audio Processing #Speech and Audio Processing
- Predicting mutational effects on protein-protein binding via a side-chain diffusion probabilistic model
2023/10/30 by Shiwei Liu, Tian Zhu, Liu, Shiwei +9 · 1 voice · 1 citation
#q-bio.BM #cs.LG #q-bio.QM
- Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning
2024/11/26 by Cao, Mingyu, Gen Li, Li, Gen +11 · 2 citations
Computer Science · #Computation and Language (cs.CL) #Engineering and Information Technology #FOS: Computer and information sciences #Machine Learning (cs.LG)
- The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter
2023/06/06 by Ajay Jaiswal, Jaiswal, Ajay, Shiwei Liu +5 · 1 citation
Computer Science · #Domain Adaptation and Few-Shot Learning #FOS: Computer and information sciences #Machine Learning (cs.LG) #Speech Recognition and Synthesis #Topic Modeling
- FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping
2024/04/05 by Ajay Jaiswal, Jaiswal, Ajay, Bodun Hu +11 · 1 citation
Computer Science · #Algorithms and Data Compression #Computation and Language (cs.CL) #FOS: Computer and information sciences #Machine Learning (cs.LG) #Natural Language Processing Techniques #Speech Recognition and Synthesis
- Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models
2025/06/23 by Zihan Wang, Rui Pan, Wang, Zihan +17 · 1 voice · 2 citations
#cs.LG #cs.CL
- ISO: An RLVR-Native Optimization Stack
2026/07/21 by Hanqing Zhu, Wenyan Cong, Zhizhou Sha +8
#cs.LG #cs.AI